Nov 11, 2024
David Miller kicks off the conversation with a summary of his life since Harvard. He went to Brown University, Providence, and then moved back to Boston, where he spent five years in Cambridge, Somerville, Arlington, and then moved to Santa Monica, California, where his wife was pursuing her fellowship. He then returned to Boston, where he has been for 20 years, minus a three-month stint in Paris, France.
Technology Inventor, Independent Contributor, and Instructor
David met his wife, Ruth Herzman Miller, in October of his freshman year at Harvard. They have three daughters, and David has spent some time as a full-time dad with each of them. He majored in mathematics at Harvard and pure mathematics at graduate school at Brown. After a pause, he worked in speech and language processing at Bolt, Beranek and Newman (BBN) Technologies in Cambridge where he worked on developing speech to text transcription and information retrieval. He went to UCLA to learn bioinformatics and worked at a bioinformatics laboratory at the Molecular Biology Institute. In 2001, he returned to Boston and worked at Aventis Pharmaceuticals, now Sanofi Aventis, applying his knowledge in lead generation informatics. He stayed at Aventis for a few years before taking some time off the workforce. In 2008, David joined Google for 16 years, primarily on the search engine. He has worked on various projects, including the Google Books project and AI Overviews. He has also spent time at Google Paris, Zurich, and Tokyo. He has also taught computer science in the context of the Girls Who Code Project, where he distributed curriculum material to numerous chapters and hosted meetups. He also worked with Microsoft TEALS (technology and learning in schools), teaching ninth and 10th grade computer science.
Inspired by French Theater
During the pandemic, revisited an interest he had discovered in Paris, French theater. He started studying French and learned about the annual Theater Festival in Avignon, France, which is the second largest Fringe Festival in the world. After visiting the festival in 2022, he decided to create a similar event in Boston. He started a limited liability corporation with knowledge of French, Boston theater scene, organizational capacity, and spare finance. The first production was performed in April 2024, and the second is set to open in November 2024. They are currently booking venues and signing contracts for their 2025-2026 season.
Google, AI, and The BERT Revolution
The conversation turns to AI, BERT, and Google. He explains that the feature of BERT was built to transform language problems into arithmetic problems, using embeddings in high-dimensional vector spaces to catch semantics. This allowed for more complex arithmetic than just adding and subtracting. The BERT Revolution, invented by Jacob Devlin and his colleagues, was used to map words to embeddings, allowing for real-world correspondence in arithmetic. This concept was later used in Google's Featured Snippets, which was revamped to use embeddings and the Bert revolution. David’s lecture at Boston University, which is titled "Natural Language Understanding, Deep Learning and the BERT Revolution" discusses the underlying mechanics of natural language processing and how it transformed problems in language into arithmetic. The BERT Revolution allowed for more complex arithmetic than just adding and subtracting, making it easier for neural networks to perform complex tasks.
The Rise of Hidden Markov Models
David talks about the state-of-the-art technology at the time, Hidden Markov models, which had a temporal aspect of a changing probability distribution. These models were based on the sequence of text, and the Bayesian reasoning was used to determine the most likely audio to come from the words. This led to the development of generative models, where words generate the audio through probabilistic models. However, Bayesian modeling has been replaced by deep neural nets in the last five years of generative AI. He mentions that, in the early days, neural networks were untrainable and unwieldy, making Hidden Markov models the Bayesian generative approach. However, deep neural networks are now used.
The Development of Neural Networks
David discusses the development of neural networks, a technology that has been around since the 1950s. The availability of more recordings for speech, text, and language models has made it more accessible on the hardware side. The core of a neural network computation is matrix multiplication, which has been addressed by Nvidia and Google with their TensorFlow units. These units have invested large amounts of money in making specialized, custom hardware for this problem, accelerating things. David talks about how algorithms have also advanced significantly since the 1950s, and mentions key factors that have aided the advancement.
Becoming an Individual Contributor at Google
David talks about how he learned the technology. He decided to become an individual contributor and studied the technology, the code, the papers, books, videos, and experiments. He spent most of the pandemic working on neural nets that eventually became the Gemini technology. David’s journey to becoming a knowledgeable and skilled individual in neural networks was a journey that took him from a theoretical interest to a practical application. He learned to make the most of the technology and its capabilities, ultimately contributing to the advancement of the field. David has faced mixed reactions to his decision to become an independent contributor at Google. While some were supportive and skeptical, others were skeptical. He talks about the advice he received, how he moved forward, the success rate of his projects, and how his career has decelerated since 2019.
Behind the Curtain of French Theater
The discussion moves to French theater and how David has become a French theater producer. He shares his journey of starting a production in Boston from scratch. To start a French theater production in Boston, David had to be integrated into the French community in Boston and the theater community in Boston. They do not create the theater but bring the original production to Boston and add subtitles. He talks about the challenges faced in securing locations, staff and equipment, and managing the production process such as hiring a director, actors, space, marketing, and logistics. He uses services like Playbill to manage administration, program design, publicity, and logistics. He is passionate about creating a new cultural institution in Boston that focuses on French theater. Boston is known for its strong ties to France and hospitals, and David aims to create a French theater festival or translate French theater into English. He works with the French American Chamber of Commerce of New England, which helps create businesses and connections in Boston.
Behind the Screen of Girls Who Code
David has worked with Girls Who Code, an after-school program that runs programs for young women interested in programming and technology. He organized a meet-up at Google's Cambridge office, where he gave a keynote speech at parent meetings, emphasizing the importance of belonging and ownership in the industry. He was able to connect with 150 teenage girls and their parents, who expressed gratitude for his message. David's involvement with Girls Who Code has led to a sense of belonging and empowerment for these young women, who are now more likely to pursue careers in the tech industry. He believes that the French language theater in Boston could potentially sustain them through a 25-year career in the industry.
Influential Harvard Professors and Courses
David shares his experiences as a TA in the math department and working with Deborah Hughes Hallet, who was running a calculus project. David's theater experience has played an ongoing role in his teaching approach, as he learned from her dedication and approach to teaching. He believes that the notion of understanding the world through teaching is a deep-rooted belief in his approach.
Timestamps:
03:59: Professional Career and Industry Experience
06:52: Non-Professional Activities and Community Involvement
11:22: Technological Advancements and AI Overview
25:07: Transition to Individual Contributor Role at Google
30:17: French Theater Project and Community Building
40:39: Impact of Girls Who Code and Teaching
45:25: Final Thoughts and Contact Information
Links:
Theater: www.frenchtheaterproject.com
Theater Club: https://frenchlibrary.org/french-library-theater-club/
Website: www.monsieurmiller.com
LinkedIn: https://www.linkedin.com/in/davidrhmiller/
David’s 2019 lecture "Natural Language Understanding, Deep Learning and the BERT Revolution" at Boston University : https://www.youtube.com/watch?v=DktFhgxynFE
Featured Non-profit
This week’s featured non-profit is the Cure San Filippo Foundation recommended by Adam Shaywitz who reports:
“Hi. I'm Adam Shaywitz, class of 1992 the featured nonprofit of this episode of The 92 report is the Cure San Filippo Foundation. This organization is dedicated to advancing treatment options for children affected by the devastating childhood dementia known as San Filippo syndrome. I am privileged to serve as a board member for the past five years. You can learn more about their work at Cure Sanfilippo foundation.org, that's one word. Cure Sanfilippo foundation. San Felippo is spelled s, a, n, f, i, L, i, p, p, O, that's 1f, 1l, and 2p Cure San Filippo foundation.org, and now here is Will Bachman with this week's episode.”
To learn more about their work visit: www.CureSanFilippoFoundation.org.