On rethinking how audio is captured, represented and retrieved in this new world of AI
Ishwarya Ananthabotla completed her BS and MS in Electrical Engineering and Computer Science from MIT. She is pursuing a PhD in the MIT Media Lab’s Responsive Environments group, exploring ways to capitalize on our knowledge of human perception, cognition, memory, and attention, to re-think traditional paradigms for audio capture, representation, and retrieval.
Madhav
Today my guest is Ishwarya Ananthabotla, Ishwarya completed her Bs and Ms in electrical engineering and computer science from MIT. She’s pursuing a PhD in MIT Media Labs responsive environments group exploring ways to capitalize on our knowledge of human perception, cognition, memory and attention to rethink traditional paradigms for audio capture, representation and retrieval. Hey, I saw you Welcome to the show. Thanks so much, mother. Thanks for having me. W MBR. 88.1.
Yeah. Can you share a little bit about like, how is that part of your life?
Ishwarya
Yeah, sure. W br is. So basically, this wasn’t pre MIT. This is definitely post MIT and as a graduate student, so
it’s now been two or two years plus and thank you. Joined so basically w MBR is the college radio station on our campus here at MIT. And you can basically sign up as a student you’re a student there’s a lot easier you just sign up to go and host a show of your choice of Country Music Show can be a talk show,
Madhav
or you associated with it like you’re on a show on that
on a daily basis weekly How does that work?
Ishwarya
Yeah, so I think I started in 2017 I’m not wrong. And I ran for you know, different season is basically a semester long right? So it follows our calendar. And so I did a few different seasons of a few different kinds of shows. The longest running one which I’m getting running this summer is called post it wall. It’s a talk show that involves into were basically interview kind of similar to both actually. Yeah, you know, interview other people in the community and ask them to share their story. I’m here at MIT, I’m in the heart of Boston and being an MIT student is something that there are a lot of really wonderful opportunities and privileges that were presented with being here on campus radio across the street from me that I can capitalize on lots of other activities, lots of people to meet and learn from. And so I’m starting to realize that if I don’t kind of seize the moment and make time for these things, now, I don’t think I will ever be able to do that.
Madhav
Well, do you like for example, to use any tool or do you sort of say, between these hours and these hours? This is dedicated for my research, or I’m trying to understand how you structure your life daily to help you do all these things that you want to do.
Ishwarya
Yeah, and I don’t know that this works for everyone or that it’s not really the right way to go about this, but I compartmentalize quite a bit. So if I actually kind of intentionally try to avoid multitasking during the workday so much, you know, maybe to the annoyance of some of my other wife, friends or so on, right, so everyone knows that I basically check and respond to WhatsApp messages or text messages.
Madhav
What is what is the story of the Daydream company? Your inclination or your love for short stories and
poetry and things like that? Yeah, come from.
Ishwarya
I’m so the Daydream company is actually also a production that we did for web or for the Radio Free. But yeah, you’re right, it actually the whole thing started because I really liked to write also, to read that too, to add innocence as a kid a love for writing short stories. poems, writing songs once in a while. And so, actually, people don’t think about this at MIT. But one of the highlights I think of my undergrad experience was humanities faculty that I got to interact with series of short story writing classes, poetry, writing classes, and so on. At some point, there were a bunch of short stories that I had written through those classes and so on. And I kind of toyed around with the idea of turning some of those and then some other ideas too short. And I was really fascinated by this idea of of radio place where you really have to kind of in that entire narrative, a story but there’s no visual
Madhav
visuals and
hold on nother ballgame. Yeah.
Ishwarya
In terms of bringing color and emotion through dialogue, delivery and through voice, but also, you know, sound effects and
the simple things right, like entrances and exits. Yeah.
Madhav
See anything? How do you think so you kind of have to show it to them quote unquote, right?
Ishwarya
I actually got this idea from from my grandmother she used to write and
Madhav
in All India Radio, something like
pink does.
This radio plays were very popular. Yeah. Before the televisions came on the scene in India, All India Radio, which is the national radio program. They would have local programs in different languages and they would have a big part of that would be place.
Ishwarya
Yeah, exactly. And I don’t know if it was All India Radio Are you something else but I do remember her obviously have no recordings of this or no, just it’s hearsay you know from from her and from people who are now generation above me but for children please and I was I was just fascinated by that idea how does one story and write it in the form of a script that can still convey the essence of the story without visuals.
Madhav
I’m switching gears a little bit let me this is sort of really fun to understand that facet of Xperia, which is sort of really fun loving and creative. And but I know there’s a lot lot more going on in your life right now. And just take us back a little bit into your journey. How how your journey to MIT started, like what got you interested in technology and why MIT more curious about what kind of got you started on that path? And at what age was it? Right from beginning? You were super curious about technology and you’re playing with gadgets your dad brought home, what was it like?
Ishwarya
Yeah, I mean, I should preface this by saying, you know, I think now if I were to put the credentials I had at that age at age number 16 or 17. I compete with the kids that are playing. Now. I don’t think I stand a chance. So fast.
wasn’t that long ago?
Madhav
It was just eight years ago,
Ishwarya
eight years ago. Yeah. So
yeah, for me, though. I, it was, I can’t say that there was this, you know, one eureka moment where I decided, oh my god, I love tech and this is what I want to do. It’s kind of a testament to how much the forces in a household are in how much the people are. Your life kind of can really shape shape your interests in a positive way. Like my, my dad and my grandfather within two years, my dad graduated from IBM or he did his masters at IBM and robotics. He was he was always fascinated with robotics.
Madhav
And for those who don’t know, I mean, I’m probably sure everybody knows about it. My dress is what you’re referring to, there we go.
Ishwarya
And he sort of had this running interest in in robotics. He, you know, given the opportunity, he would have done a PhD or, you know, continued to work as a research scientist in that area, but unfortunately, starting a slightly different career, but he kind of build that environment for us at home, there was always the robot gadgets they would come for Christmas presents. Well, there’s a product He would encourage us to do whenever he would travel as a consultant he would is models of bridges or famous buildings from the area and then my sister and I would enjoy putting them together. Yeah, I think we have most of them. So hang around the house. And so these all kind of these little steps that, you know, not only one chose a young girl, a kid or younger led to somewhere that
could apply there. It feels
also to that.
Perhaps this
school doesn’t always get you back.
So
Madhav
that’s a very Yeah, that that itself is that like that small bow going off moment. Like perhaps I could do it. It’s not like Yes, I know. I’m going to Doing this and this is what it is. It’s not a big revelation. It’s more of a Yeah, perhaps I could do this this is an option for me.
Ishwarya
Yeah, this idea that it’s an option and perhaps contrary to societal norms this year, perhaps I can succeed at it. Yeah.
Good fight.
Madhav
Always a fighter.
Ishwarya
I was a frankly, yeah, pretty pretty rowdy local.
Kid Middle School agers
Madhav
beat the boys and girls up in the class.
Ishwarya
very talkative, very high energy. I think.
One of the things I’m really grateful for I didn’t go to one of these two private school or to a charter school ending on public high school and Long Island.
Madhav
Kings Park is our Kings Park High School.
Ishwarya
Kings Park High School. Kids Park, New York. Yeah, eastern end of Long Island. They had a few I should really good but small programs for kind of pushing kids forward who are academically motivated or who are interested in doing research.
Madhav
And so they, I was gonna say stem, but it’s actually more than that.
Ishwarya
No, it’s actually research specifically, pick up a handful of kids. End of middle school and if you have good grades and you’re motivated, they would put you in this program where they basically pair you with nearby universities. And you can basically be a high school intern and get to do some basic tasks for them slowly learn the ropes and then your senior year of high school, take on some slightly more involved project, write a paper and so on.
And so I think that was really the major stepping stone JOHN to
Madhav
and this was just part of your regular public school but they had a separate like a program that was created to encourage people who are inclined and more to do that. But you had the good fortune of having such mentors in school was one such I think I might be Miss pronouncing the name of Jim shock
Ishwarya
yeah
oh man you’re really good at googling
this information was public
Madhav
wow it did I say that name right. Jinja
Ishwarya
chain choke. Yeah.
Madhav
Oh, and what how was she like I did see her good remarks about how he deserves this when for internal science fair competition or something of that sort. Where she one of your mentors.
Ishwarya
She was. She was one of those basically honored to teachers that run this program. And they kind of split up the students across the four years in high school and mentor them through the program. She was also my high school physics teacher. They just like a double double role that they play. Right. You know, if I had to say, if I had to, you know, pick a handful of the most influential human beings that I have encountered are the people that have single handedly shaped. Yeah,
she’s probably
Madhav
I heard many stories from many people like that where there was this lady there was this teacher there was this person in my life when I was 17, whatever, that really shaped and changed the way I looked at things, the perspective shift. And so you would consider Jane show as one of those how in particular, did it affect you positively
Ishwarya
You know, it’s not that she necessarily introduced me to a new discipline or taught me to think about things differently, but she really kind of saw a spark in me or saw the potential that I may be had at that time, and really pushed me the most of it. which involves, obviously the comics and motivating me but also her being very involved in reading everything that I would every draft of a proposal or paper, right, staying after school for two or three hours with me and another student as we practice our presentations for these missions.
Madhav
Yeah, that’s, that’s intense. That’s insane, actually, I mean, to have that kind of deep interest in a student to push them because they can see the potential That’s insane. Well,
Ishwarya
it’s very interesting because it’s sort of this idea of exercise and detachment, right? Because they put so much sort of invest so much time and effort into a student. And after four years, I mean, maybe these people send an email once in a while. Gone.
Madhav
You said it, did you send her an email?
Ishwarya
It was it was a few years ago. It was actually an undergrad. I sent her an email when I had won the research competition here. So I wanted to thank her for being a part of the whole
Madhav
Were there any things that guided you have to in your journey to get into MIT?
Ishwarya
Yeah, undergrad admissions is such
a blurry
Madhav
to black box, the
Ishwarya
big black box. Yeah, it’s really hard to say and also because it’s been it’s been Nearly a decade ago embarrassing to say. But I think if I had to give one advice, you know, one piece of advice about this phase now it’s I think being the quote unquote, kind of breath first well rounded is is kind of overrated. If I from the people around me when I first met these people in undergrad excluding myself for a minute I would find these these amazing individuals that were there their strong suits or their skill sets, the things that kind of got them into my team or maybe these these three things they were really good at math competitions, they were all star oboe player and some something else. But the point is, they’re at these handful of things that they really took really far. And then some sort of had wonderful parents that were willing to, you know, put me into these things and
lucky
Madhav
Nice, nice.
And you’ve also went to grad school at MIT,
Ishwarya
I work more on the computational audio and audio signal processing side of things. Sort of a few different things going on. But at high level, it’s this idea of cognitive scientists and psychologists for a long time have given us models of perception and cognition In addition, right? So way that may perceive sound both sort of at the interface to hearing but also in terms of memory and all these other higher level complicated causes. We haven’t yet taken you know, this this information, these on this understanding and applied it to the way that we capture audio
or the way that we replay and listen to audio.
Technology hasn’t met the humanistic side of things yet.
Madhav
Before my break that and I’m unpack that a little bit. That’s a lot for me. So, what do you mean by like, we haven’t tapped how these audios perceived by humans and how, for example, I think you, you published a paper around the intrinsic memorability of everyday sounds. Curious like, what? Can you unpack that? That sentence is kind of intimidating to me?
Ishwarya
Sure, I guess simply stated.
Not everything that will be here or that we’re presented with as a sound. Not everything we hear we perceive, and not everything we perceive we remember, right? But if I were to stick a microphone in that same in the same setting, in this hypothetical context, you’re going to record everything,
Madhav
everything, but it’s still just the sound
Ishwarya
every where we define sound as a variation in the air pressure, right. So all of that That transpires is captured, that’s what a recorders microphone your eyes standing in the place of that microphone will, quote unquote not record all of that. I see my brain works right. I
will take an exit survey
later An hour
Madhav
later, you probably missed 30% or 50% of what was
Ishwarya
actually missed 80% of
this this sort of thing, which is like perhaps a transcription task or surveillance task is not where you want to be using kind of calling it like cognitive compression or memorization. Right? Yeah. You’re recording this to obtain some sort of specific information that you’d like and we’d like to go back to and you know exactly what you want to go back to for whatever reason. We’re talking about people’s audio life loving is now this thing where people will just wear around a recorder or a camera four hours and a half thirds The goal, what you want to get out of that is something aesthetic for the experiential, not exactly who was wearing who would see. The replay that audio and just kind of go at the sound with the style of go back comes back, when you hear Oh, my grandmother was cooking in the kitchen, she was humming something, I hear the pressure cooker going off. And it reminds me of that setting, or this recording is so indicative of the lobby of this building, because you know, the door closes in a particular way. And I hear it all the time. And so the idea is to kind of apply these set of algorithms to create these different spiritual outcomes based on different settings that are applied, that are the algorithms that are applied to this audio. So if we model attention really well, and then we run, sort of just pretend that that attention blackbox and we can feed all the audio that we’ve recorded over our Through this black box, we can do one of two things, we can take all of the audio that you might attend to. Or we can do the exact opposite with just like all the things that you’re likely not doing to be a great track for studying or for sleeping. Well.
Madhav
Could you touch on how some of those advances in big data? And collecting? Like, there’s more and more audio being recorded now than ever before? Right. Is your research sort of using some of these deep learning frameworks and things like that also, or is it not
related at all?
Ishwarya
No, it’s, you know, built pretty significantly on on deep learning paradigms, right? A lot of the sort of computational implementations of these models of memory and attention so on are enabled. Kind of enabled being running the business running in real time? And because of how quickly we can now do insurance model these higher level principles via deep learning, basically. So yeah, it is it is extremely related. But what you touched upon is actually exactly the problem space that I’m trying to tackle. We know that basically, nowadays because of you know, IoT enabled, the more audio that people will record for monitoring purposes for whatever it is, the less likely they are to interact with. Yeah, more there is that’s true. Like,
Madhav
we all have like a million pictures on a phone taken, but you probably never look at them again. Yeah, yeah, the audio
Ishwarya
and audio is worse because
Madhav
actually You got it.
Ishwarya
Right. But it’s, it’s because of because of that because you can’t see it. It’s harder for people to If you want to go back and find one thing in particular, that’s, you know, that’s great. But people don’t necessarily know how to navigate through hours and hours of audio recordings. Right. and meaningful fashion can have two ways to go about artificial intelligence nowadays, right? there’s kind of two clans. One is kind of the majority is that the group of researchers and scientists are working regarding that, like a particular cognitive level of or strive to emulate. At the cognitive level of a three year old, classic life is
Madhav
actually right. Yeah.
Ishwarya
Hey, john, rock music so on you, and it’s important that we start there. But the objective is to basically simply compete with or use the standard for the human level and ensure that you know and use this as a baseline for
you as well. Right? There are
a not so vocal but a smaller kind of group of researchers more on the kind of on the border between cognitive science and AI, who are looking to work on artificial intelligence problems as a way of gaining more insight. The human mind actually works. And to me, I feel like that is a the more interesting kind of side of the problem, but also significantly more difficult
Madhav
to understand and deconstruct the workings of the brain. Yeah.
Ishwarya
Yeah, exactly. So,
Madhav
taking a positive approach to it. You’re right. I mean, there’s so much that we can actually deconstruct and learn and maybe address some of the challenges around dementia and whatnot. Let me sorry for jumping all over a little bit. Bear with me, but, like going back, because you brought up this idea of understanding the mind and the brain and other things. What tactics do you use to sort of
maybe brain hacks?
Ishwarya
Yeah. I think for me undergrad was was probably that time, sort of the time of more stress in my life, had to do with the struggles of leaving home for the first time and being in a new place and living in a dorm. All the social pressures as well as trying to keep your grades up. I don’t know I think I resort, I would resort to the to the simplest thing. So I would involve for me a lot of music, in terms of just taking time out discipline. allows me to pick up the pieces a little bit and kind of revisit a mental thread that has been particularly interesting.
Madhav
Do you also practice any, like meditation or any of that sort,
Ishwarya
I would actually kind of do meditation in an interesting way, which is, so for me kind of just sitting down and sitting in silence was really hard because I don’t spend the majority of my day sitting in one place thinking, right? And then kind of doing the same thing as often, as we were right on the Charles River here. Yeah. I would often walk to the river, stand by the River Walk.
Madhav
So that’s an interesting point. To me. It’s like meditation doesn’t necessarily mean sitting and close your eyes, but it’s it could be something else that changes like tasks which switching if you’re sitting all day, it’s a great day. Yeah, just get out and take a walk can smell the roses.
Ishwarya
Yeah.
Great reminder of things that are bigger than new things that are either always moving or always present. Right? And to kind of keep that juxtaposed with whatever, whatever’s bothering you at that period of time. Is this a good reminder? On the spiritual?
Madhav
Yep. Or is this something that you do as a routine? Is there something that you would do? I know people I’m sure you’ve heard of people say up. The first thing I do after I wake up. I just meditate for three minutes or a chant santen transcendental, whichever Are you know, I don’t look at the phone for the first hour are the routines that you’ve picked up over time that
you try and stick to
Ishwarya
So this this taking a walk by the river thing is this one thing but this sounds really silly but every morning after I showered whenever I have a cup of coffee, but during the time that I have a cup of coffee,
Madhav
go Joe is always a good thing. Is there filter coffee or South Indian filter coffee?
Ishwarya
Yeah, something like that.
Indian coffee. Yes.
Yeah, the point is not the coffee itself. When I’m when I’m taking the time to drink my coffee every morning and it actually lasts for 15 minutes or 20 minutes is like, force myself not to engage with my phone. And I’m most fresh and important for me to sit down with my thoughts.
Madhav
Do you jot them down? Or is there something like do you actually write the thoughts down? Or how do you record them?
Ishwarya
during that period of time? It’s literally just me in silence, taking
as long as my coffee left.
I don’t read anything. I don’t listen to anything. talk to anyone. Just my focus time with my
once a day, I really have it again,
Madhav
for lemon 15 minutes. Not too far. Not too long. So people don’t have to complain. I don’t have that kind of time. I’m too busy. Just I’m just trying to not do any other things. just sip coffee and let your thoughts so it’s sort of like being mindful or just observing what’s coming across. Yeah, right. And on that note of mindfulness and being Are there people books that sort of stand out to you that
Ishwarya
So interestingly enough,
I like to read and write a lot, but I often actually kind of like the plague I stay away from these kind of books that are, you know, talk about personality traits. Or, for me, it’s just really hard for you to sit down and read those things and process
for me to sort of didactic
learning what are the backbone of books
about these things sometimes resonate with me and I’d much rather be offered by reading poetry or whatever it is, but
Madhav
anything that stands out like any line or a particular poem or put
Ishwarya
any lines but yeah, sort of actually the very same space of poets. I read like Wordsworth, Elliot, on the Sufi side of things I A lot of translations of
copies of militia
Madhav
in terms of the philosophy of one of the things that I’d like to learn or, you know, talk about, and see what where your mind is at in terms of loving kindness in what you do on a daily basis. However frustrated and crazy your life might be. How do you bring that kind of calmness or kindness in the way you say things or the way you act?
Ishwarya
I think for me, it’s kind of too too precious perspectives on the problem. One is the simple idea that every action has a reaction and so somebody is behaving a particular way.
Madhav
Of course, I think
Ishwarya
if someone is behaving a particular way, a particular way, it’s likely motivated. by some other event or something else that has transpired in their day or in their life or some other contexts that is hidden to me.
Perhaps if I knew the full story, there would be some sort of
way for me to connect the dots. But for the most part, we don’t have that information, but kind of that thought process allows me to give, I tried to always go as a sort of more rational way of thinking about why we should kind of deliver kindness or compassionate despite what we’re getting in return.
Maybe one thing and
I know this sounds cliche, but to me, there’s something really important which is this idea that if I believe very strongly in a higher being and this idea of God and what God really is present in me and present, every human being that I will track with, I do offensiveness, I react rudely and politely interacting with a being much more divine than I can kind of proceed.
Alright, keep that in mind. But
Madhav
as long as you’re conscious of and have the thought process as a second nature, it helps. And just to touch on, like, spirituality, how does that play in to this? Your take on practical spirituality in daily life? versus spirituality as a ritual?
Ishwarya
Yeah, I mean, I have time that I set aside. I mean, I definitely carve out tons of time in a week or in a month to do some of these more maybe involved ritualistic things, going to the temple or going to budget these things. I think that I actively enjoy doing the best you take the limit. For me spirituality is something that moves with me, right? It’s this this idea of being aware of the presence of God or higher being Your day of realizing that all of your actions have reactions.
Madhav
Just a few rapid fire questions, you don’t have to answer them in an epic manner, but you can take time to answer them. But just what do you want the world to remember? Remember you for
Ishwarya
someone who was a sort of medium or a vehicle for spreading joy, which could walk into a room and just put smiles on the faces of the people around if people remember me for that, then I think I
lived all I want to do.
Madhav
Is there someone you admire and aspire to be like?
Ishwarya
My grandmother, her her outlook on life is is is very simple. This very simplistic view of the world that good brings good and Prince.
Madhav
Super simple. We make it so complicated. It’s just super simple
Ishwarya
and this view of the world That’s very much guided by the tenants of our faith snapshots into mythology, and to our epics to our literature, sort of guiding principles for this little black and white. That is, that is the way she views life.
Madhav
You said you would drop everything for a jam session.
Ishwarya
Absolutely. I mean, drop everything in the sense of not give up my degree or whatever, but good job, whatever it is I’m doing at that period of time. coming to me is sitting there and singing or playing instrument for yourself is important meditative experience for me, but I think the joy of music comes from cherry.
Madhav
sharing with a lot of people.
Ishwarya
Yeah, kind of participatory.
Madhav
Yes. Sharing the joy with others. That’s what makes it
Ishwarya
happy. Yeah, I think the energy you get out of that kind of synergy of musicians together in a room just just having fun with the serving not With any structure, this is a lie. Is is, is an energy that is high for me that I can be.
Madhav
If you could write something on the full moon that the whole world can see, what would you write,
Ishwarya
can write something for the whole world to see.
I wish I had some time to think about,
Madhav
come back to it later, you can come back to it.
Maybe I’ll jump to the next one. Do you feel sometimes that you’re in a box, and you want to get out of it to go to the next level? Yeah.
Ishwarya
That’s true. I think in the future for
being in Boston being
a Top Gear technical institution. feel like you’re in an echo chamber. You’re hearing back a lot of who you are and what you believe in. Sometimes that’s that’s nice. That’s comforting. have obviously been in the same physical place in the same institution now for so long. So we’re looking forward to growing in that sense. Maybe after graduation, moving away from Boston.
Madhav
Did you have challenges in school growing up as a question?
Ishwarya
Sure. I mean, many of us that went to public high schools in suburbia and New York or in the US, this is something that people are familiar with, which is that you kind of lead this double life. You. You go to school you may provide.
Yeah.
I think except for one other student. I don’t think there any other televisions near the majority was Caucasian. as well. I don’t think we have many African American students are Hispanic. And so you are one person in that setting. There’s things that you can’t say are certain things that you just don’t bring up in conversation. The clock strikes five Whatever it is, and you come home and you with a family and it’s like you’ve been transplanted to India for speaking, eating South Indian food, we’re dressed in a particular way or watching Indian movies, whatever it might be.
That’s, that’s all over again. So caught me can prove frustrating occasionally maybe. But I think in some ways it was. It was a really good experience both of these things
Madhav
in enriching Actually, yeah, it’s, it’s, it’s just not one, it’s two and two different, completely different way of looking at things in some aspects. Would you What would you write on a full moon, if you could write something, it could be a word, it could be a pause. It could be a slogan, it could be whatever just a message that you want to share with people.
Ishwarya
You know, it doesn’t have to be something that is like you know, meaningful advice, but
Madhav
just something that goofy
Ishwarya
life is a song Sing it
Madhav
awesome. Life is a song Sing it is it that you truly believe in that maybe very few people on the planet believe in
Ishwarya
something but it’s not really tech oriented or it
Madhav
can be completely Yeah.
Ishwarya
Second personal, very strong and personal belief being family is a gift from God. Staying family is an active choice.
Madhav
That’s a great place to end
Transcribed by https://otter.ai
Show Notes & Links
- WMBR MIT https://wmbr.org
- http://www.mit.edu/~ishwarya/
- https://patch.com/new-york/kingspark/top-of-her-class-ishwarya-ananthabhotla
- Intrinsic Memorability of Everyday Sounds https://arxiv.org/abs/1811.07082
- Life is a song, sing it