The Memory of All That: George Gershwin, Kay Swift, and My Family's Legacy of Infidelities

The Memory of All That is Katharine Weber’s memoir of her extraordinary family. 

Her maternal grandmother, Kay Swift, was known both for her own music (she was the first woman to compose the score to a hit Broadway show, Fine and Dandy) and for her ten-year romance with George Gershwin. Their love affair began during Swift’s marriage to James Paul Warburg, the multitalented banker and economist who advised (and feuded with) FDR. Weber creates an intriguing and intimate group portrait of the renowned Warburg family, from her great-great-uncle, the eccentric art historian Aby Warburg, whose madness inspired modern theories of iconography, to her great-grandfather Paul M. Warburg, the architect of the Federal Reserve System whose unheeded warnings about the stock-market crash of 1929 made him “the Cassandra of Wall Street.” 

As she throws new light on her beloved grandmother’s life and many amours, Weber also considers the role the psychoanalyst Gregory Zilboorg played in her family history, along with the ways the Warburg family has been as celebrated for its accomplishments as it has been vilified over the years by countless conspiracy theorists (from Henry Ford to Louis Farrakhan), who labeled Paul Warburg the ringleader of the so-called international Jewish banking conspiracy. 

Her mother, Andrea Swift Warburg, married Sidney Kaufman, but their unlikely union, Weber believes, was a direct consequence of George Gershwin’s looming presence in the Warburg family. A notorious womanizer, Weber’s father was a peripatetic filmmaker who made propaganda and training films for the OSS during World War II before producing the first movie with smells, the regrettable flop that was AromaRama. He was as much an enigma to his daughter as he was to the FBI, which had him under surveillance for more than forty years, and even noted Katharine’s birth in a memo to J. Edgar Hoover.

Colorful, evocative, insightful, and very funny, The Memory of All That is an enthralling look at a tremendously influential—and highly eccentric—family, as well as a consideration of how their stories, with their myriad layers of truth and fiction, have both provoked and influenced one of our most prodigiously gifted writers.

"Novelist Weber (Triangle) here paints a wry and engaging portrait of a powerful, talented, but troubled family. She relays memories of her father, Sidney Kaufman, a self-mythologizing filmmaker, and Andrea Swift, her self-absorbed mother, who retreated into photography and Angela Thirkell novels, and weaves them into her familial history: on her father's side, refugees from Russia and Eastern Europe; on her mother's, the German-Jewish Warburg banking dynasty. While the affair between her maternal grandmother, composer Kay Swift, and George Gershwin takes center stage (Weber's title derives from an Ira Gershwin lyric), Weber packs in other celebrated names related by blood or association. The most touching passages describe the impact of unavailable adults on Weber (she was left alone for five days on a film set) and Weber's relationship with Swift, who took her to Broadway shows, Central Park, and Schrafft's soda fountain. (July)" Publishers Weekly Copyright PWxyz, LLC. All rights reserved.

 "To be a writer born into an illustrious and complex family is both a burden and a gift.  In THE MEMORY OF ALL THAT, Katharine Weber trains her novelist's eye and penetrating intelligence upon what may be her greatest subject: her own family's history as it stretches back, generation after fascinating generation.  Her achievement here is a literary one, to be sure--but even more than the beautiful, elegant story contained in these pages, I am in awe of the strength, tenacity and courage it took to rise up out of this fabled cast of characters and write one of the most powerful memoirs about inheritance I have ever read."

--Dani Shapiro, author of Black and White