How To Design Generative AI Features For Adobe Acrobat (& Break Into Machine Learning Engineering Roles) - w/ Nikhil transcript

if I want to forge a career in machine learning and or artificial intelligence what should I focus on should I look at building stuff or should I upskill myself doing LeetCode. So firstly it's not just one path what did ChatGPT do that made it the biggest name in Tech last year machine learning is almost everything is about research some of my friends they don't come from traditional research backgrounds and neither do they have any inclination to do any research are there any courses that would be good for people that are passionate about machine learning I had to reach out to the hiring manager for that role and then I straight away got the interview I mean that's honestly a flex right there how do you make good UI/UX specifically for generative AI right now we are working on chatbot-like AI assistant for

the Adobe Acrobat what was your interview experience like how did you end up at Adobe there was a time last year where they were paying like 300K for prompt engineers and everyone was so surprised Welcome to the Ready Set Do podcast where we learn from journeys of not experts who are just two steps ahead of us I'm Naman Pandey and in this episode the featured not-expert is Nikhil Nikhil is a senior machine learning engineer at Adobe currently working on building cutting-edge generative AI features on Adobe Acrobat throughout our discussion Nikhil references his experiences and learnings from working on document intelligence for almost 5 years now to walk us through the best ways for current students and professionals to get hired for their first machine learning engineer role I think you especially enjoy our deep dive

on the best strategies to set oneself apart by building machine learning projects and how that ties into the interview process for most machine learning engineering roles Nikhil also takes us through a day in the life at Adobe and what the future holds for the generative AI capabilities of Adobe Acrobat which is the most popular PDF tool worldwide I also thought Nikhil's take on the impact of AI in the not so distant future was particularly interesting in keeping with our theme of learning from somebody that's just two steps ahead of us instead of an expert my goal with this episode is to highlight Nikhil's incredible journey and how he was able to forge a career in this niche of document intelligence this is the Ready Set Do podcast and to support it please subscribe to the YouTube channel and or leave me a

five-star rating on Spotify or any other podcast app of your choice you can also check out the links in description for other ways to support me some being more direct than others and now without any further ado, my friends, here's Nikhil welcome hi Naman it's good to talk to you let me begin with the question that's really on well I was going to say everybody's mind but seeing as that there's nobody else here just my mind um and the question is if if I'm a current engineer or I'm a current student say studying computer science maybe B.Tech or master's maybe it doesn't matter and if I want to you know Forge my way into a career in machine learning and or artificial intelligence should I you know what should I focus on I guess is the question should I look at more projects and that type of you know building stuff

or should I upskill myself in terms of learning new technologies and doing LeetCode? Which approach would you prefer or is it a mix of both of them yeah um I can talk on that so uh firstly it's not just one path it would always be a mix you have to attack the problem from all the directions so uh so what I mean by that is uh you have to have good coding skills because and not just coding skills you should also have uh subject matter expertise where you get to learn about more about ML and deep learning and um uh you have to do multiple courses to build some theoretical knowledge right so but theoretical knowledge does not take you all the way so I would say that uh apart from courses you also have to build something using the knowledge that you have

learned from the courses and that way you'll also improve your software engineering skills, integration skills application skills and then overall ML knowledge in action so from what you're saying it sounds like if somebody were to just focus on algorithms and data structures that would give them the base to then later go and start attacking machine learning specific topics uh you know specifically is that kind of what you're saying yes I mean data structures and algorithms are important so a machine learning engineer has to be a good software engineer as well so okay one has to be a good coder. It's it's it's related it's very related so uh if you have good software engineering skills that'll be a base to start for the coding part but then you can uh learn about ML algorithms and things

like that and then it it'll become easy when you are building something makes sense that is what I thought um except then I was introduced to this notion at least I don't know anybody personally that does this but I know there are people online that talk about you know just really just hacking they're like screw all of this stuff who will learn uh you know all of these algorithm stuff let me just Cobble together a project you know like just hack together from taken from multiple places you know a workable prototype or a working prototype of anything really that solves a problem you could have one that generates recommendations whatever it's not very hard to build and and I would argue that you don't really need a lot of um you know coding skills to do that you need some but you probably don't need to understand how like the algorithm

works if you're trying to just Cobble together a project do you think that's still a sustainable way to learn machine learning or would you like not agree with that um I mean that's actually the best way because if you're building something right if you're building something and which means that you're learning it from the other side uh so not in the traditional way where you learn theory and then Implement uh if you're building something and learning hacking the way around building an application you will obviously have to learn something while you're building something so which is enough uh for you to build but now uh even though you feel like you hacked it around internally you you you will learn some things not everything but at least few things how do you connect things how do you uh do things for the rest of the things you

can definitely use ChatGPT or any sort of large language uh uh interfaces you know to um to get it like because everything is available for free now um and there's you can always hack your way around but again let's say that you you want to build something from ground up there's nothing you can refer to some new application uh that is when uh I think Basics will help uh so it's it's again a trade-off so what you are trying to build is it is is some reusable components already exist in the market that I can use open source things that I can use or is it something that I have to build from scratch so it's completely on the use case I see so what you're laying out is it's fine to hack you know hack through your way and build something but eventually if you do enough of that you will eventually reach

a point where you will need to go back to basics and learn the foundation without which you will not be able to go that far like just building one project isn't most people's goals you like you would they would want to continue building and obviously eventually maybe get employed somewhere and for that reason or for if that is the goal they would probably benefit from having that overall knowledge not just the theoretical part not just the hacking part but some mixture of both of them will probably be most beneficial would you agree yeah so one more thing I wanted to add is um uh so I live in SF and then there are a lot of hackathons happen here and then I see a lot of students lot of uh working people come there and then uh they don't have a specific domain knowledge there they might come from data analysis background

they might come from software engineering background but the hackathons are related to let's say ML uh they can't just learn it and then do it uh on the same day so it's like a one day hackathon so now this is where the hacking part starts so then uh as like one individual will go uh find the references and then connect different things to make it work by the end of the day so I would say it's a good way to get your product or prototype done to show to someone to show to some uh investors or anyone but uh again end of the day it's it's just a prototype MH makes sense and I guess segue to kind of the question I had for you so you personally when you were coming up um which approach did you use of the two that we discussed and maybe you can use that as a a way to also talk about you know your career trajectory and your

career path that has seen you come to Adobe at this point, where you are currently yeah so I started uh um when I was in bachelor's back in undergrad um uh back in India so that is when I started my um uh data science journey so mine was more in a traditional way in the initial days um but yeah let let me let me walk you through uh what happened um so after my undergrad I uh had some of the projects in ML back then uh sorry and was your undergrad just in computer science or was it more no uh so my undergrad was in electronics and communication surprisingly I was told that uh if you take electronics and communication you can go either way it's true I did hear that also yeah but surprisingly it didn't work out that way but what happened was I developed an interest in ML because my major project in my

undergrad was in ML and then that is what led me uh to uh get placed in a company and to a team where I actually started as a data scientist / machine learning engineer wow that's incredible so for me I would say the starting point was the uh major project that I've done um back in 2017-2018 uh so that is when 2018 is when I started my first job as a data scientist I see and where was that as where were you working so I was working with Reliance back then uh Reliance Industries um as a machine learning engineer uh so yeah that was an amazing team and the uh that is when the machine learning started to Boom actually uh the roles of data scientist and machine learning engineer came and then it was rapidly increasing got it and are you willing to talk about you know just a day in the life what type of projects

you were working on like what was going on maybe was it a tool or product that I would know or you know one of some of our listeners would be aware of yeah so back in the day um U so I worked mostly in natural language processing and computer vision just to go into depth kind of projects that worked on is mostly on the automating um um uh kind of a lot of document stuff and I worked extensively on document AI side so kind of uh automating extracting some of the important entities doing some classification building some classification models building some recommendation models and building an MLOps pipeline uh for these models so this would be like a traditional uh work that I've done back then um so MLOps was just getting started that time uh so it's all completely new there were no clear standards for how to build

these pipelines how do you maintain we had to come up with our own approaches the tools were actually in early stages so it was it was amazing I I felt like I worked in a startup rather than in a big company that's so interesting I didn't even realize Reliance does a lot of document stuff so that's definitely news to me there are a bunch of teams um uh so and Reliance is not small right like Reliance is associated with multiple small startups vendors everyone so all the businesses that they have everyone is going digital like uh so even the archives they had uh so they had to go digital and everything uh so everything is now digitized so apart from the document AI I also worked on something similar uh creating your own um photo platforms like similar to like Google Photos uh but it's kind of internal to

Reliance so similar to that I built something where the features are very interesting like okay face recognition you know search captioning um clustering of faces uh search over the images these are some of the interesting things done there that sounds amazing yeah and definitely I don't think it would come as a surprise or sorry I mean it would come as a surprise I think for a lot of our listeners that these types of projects are you know were worked on uh in Reliance at the time and now I can only imagine what type of projects must go on there because they've become even bigger and even better right right I think there should be a lot going on the Jio side of things now absolutely yeah it does sound like the you know buzzword of our times for sure so um what did you do

next then I I presume you went to the US for your master's what was that in and what was that experience like yeah so I worked with Reliance for two and a half years and then switched to another company. I worked there for a short time but then I got a chance to move to the US through a master's I did my masters in computer science uh from University at Buffalo, New York um so it was like a one and a half year course so uh pretty short so the main goal here was to just transition to the US and get a good job in the US so that's the whole idea and also if possible getting to good research University for the computer science so machine learning is almost everything is about research um so if you have a good research profile you get the job easily uh in in ML so that's one

thing I knew before uh so I always wanted to get into good university where there's like a balance between research and cost makes sense um some of my friends have you know they've expressed interest in getting master's degrees from the US except they don't come from traditional research backgrounds and neither do they have any inclination to do any research do you think there is a market for such people or maybe there are there any courses that you know of that would be good for people that you know like this that are passionate about machine learning but are just not passionate about doing research or you know publishing papers yeah so I can give my example so I I did research like I've done so ML is more about anyway research research new techniques, experiments till you get uh

sustainable results and then uh you take it forward uh so we used to come up with a lot of new techniques but no one in India encouraged us to publish papers in the team so uh that was one drawback um if I had to go back why was that do you know uh it's because the emphasis was just to kind of uh deliver the results deliver the code deliver the application to the users and it was never mostly about okay we as a team can can publish the paper uh that would help but because publishing also there are separate research teams actually uh so that does the research work but as part of uh generally in a few other companies uh people do the research and people also deploy the applications and develop applications so they do end to end so but I never got a chance to publish a paper when I was with Reliance

or any other company before um so that way I didn't have that research background uh or publications but I I had to take a different approach I made sure that I built good projects so I had a good portfolio that will eventually help so if I had to suggest I would definitely suggest building good projects if you are not into the research side of things build good applications with some business outcome business value so that you can present yourself to some uh to recruiters or anyone um because there are two again two phases one on the research as I told you and one on the application development and deployment you can always apply to ML uh engineer or MLOps engineer or uh uh uh forward deployment engineer things like this so these are mostly towards the uh application development side that's really helpful

context um yeah I can I'm happy to have covered that part because it is indeed a question that comes up often um yeah so then moving on uh what did you do next um I know you were you were at you know doing a co-op at a startup what was that experience like and what were you working on at the time yeah so um with the co-op um I actually got an internship and then that got converted to co-op so internship I can do full-time like 40 hours a week but co-op is like just 20 hours a week is something I could do so the main problem statement that I worked there was mostly on uh again document classification and building entity extraction and question answering pipeline so think about it like building a chatbot uh for your use case um so there are a bunch of insurance documents or or legal documents that I had um and

I want to answer a question uh user might ask any question and then uh it has to get answered and then you need to uh show them where the answer is coming from from a bunch of documents that they upload so this is a simple use case of a chatbot right uh but the problem was that the data was so huge and then this domain was pretty new insurance domain uh so there was not much innovation in the insurance domain but that is where I I thought okay this uh startup was based on insurance so so that is why I was like okay let's try exploring this this part and see how it goes got it so how did that go it was fine uh it was fine I was able to uh deliver uh better but um one it was very early stages back uh last year uh because ChatGPT had just been released and um um and everything whoever is developing chatbots or whoever is developing LLM

based applications skyrocketed like crazy uh so and the whole GenAI wave started to boom. GenAI was there before ChatGPT too, but yes, GenAI is generative AI, which means you generate something like a token or create an image. But the conversational you know like wrapping around it I guess that was that there as well I'm genuinely curious I'm not trying to you know like doubt you so no it's there from 2017 or 2018 so when things started like Transformers uh BERT-based Transformers, and then other um uh Technologies came in other models came in so these language models are all generative models so uh

so you try to just predict the next token basically so in your text so that is what it is so they were there for a long time but it's just that infrastructure was not up to the mark and then the model training data was not up to the mark and um and lot of other things but now with ChatGPT it's it's so quick that you can develop real-time applications uh and you don't need to wait for that for the answer like a lot of yeah that's so interesting I guess the the part that is still not clear to me is what exactly did ChatGPT do that nobody else had done before them because you mentioned if generative AI was still around everything was around so were they just credited with the infrastructure did they just happen to get lucky like I guess I'm trying to ask what did ChatGPT do that made it the

biggest name in Tech last year yeah I I would talk as a generic point of view uh because I am not from OpenAI I can't talk no but you're way closer to that than I will ever be so I'm just curious to get your thoughts definitely so first thing that happened over the time was uh a a shift with the infrastructure the GPUs the compute um you know how Nvidia came up with a lot of GPUs and compute power and then uh in parallel, OpenAI was gathering huge data sets uh from from open source all over the world um and then collecting more and more data training huge models so earlier in initial days BERT and everything they had like parameters every model has few parameters right uh while training so these are called uh model parameters or uh so they have like 2 billion 10 10 billion to 100

billion things like that so as the number of parameters increases the model size increases now let's imagine back in 2018 if someone wants to develop a model with uh uh let's say uh 3 billion or two billion parameters it's going to be so computationally expensive uh because of the training process because of the hosting process because of the serving process what I mean by serving is if some user asks uh something it has the model has to serve like give answers and that's not that's not that wasn't possible in real time back then because of because there were no good GPUs back then so as the time progressed according to Moore's Law if you are aware of that um uh so as GPUs improved the compute power increased in 2021 2022 is when uh Nvidia boomed as well in terms of GPU provisioning and that is when OpenAI uh

also trained their models over the Nvidia GPUs A100 and then they deployed them on on huge clusters of GPUs serving in real time so that is why when you type a question you know in a ChatGPT you get the tokens instantly just like that that's so interesting that it was you know not just them being at the right place at the right time but really sounds like so many external factors that just happened to all come together at the right time for them to you know deliver this really just sensation I feel like in my time at least or in our times I don't think we have witnessed or maybe I can I should just speak for myself but you know since the iPhone it it this really feels or smartphone I should say um this really feels like the next Frontier of the you know big big like landmark tech

Innovation would you agree I'm curious to hear your take on that I would definitely agree on it uh because um smartphone is one thing and then now um we have I I I I assumed we would reach this point in like 10 years in 2018 but it was just cut in half like in five years we were already there um so the amount of speed uh um with which we reached here is amazing I would say and ChatGPT is a game changer and OpenAI is a game changer actually truly indeed um yeah I guess continuing where you know sorry for that sidetrack uh but at least for me that was really really helpful because really like this whole time I had no idea how or why you know what you went over had happened so that's really helpful context but from what it sounds like through your career trajectory so far it feels like a no-brainer that you work at Adobe I know

Adobe does stuff other than document stuff but because you know what you from what you laid out everything you've done is has been around document classification NLP like machine learning and documents essentially is the vibe that I'm getting so how did that come to be though I don't want to assume a lot so what was the interview experience like what was your like how did you end up at Adobe essentially um yeah I was actually fascinated by a fact because I was working too much in document AI um I was always curious like who was the inventor of the PDF and it turns out it was Adobe okay I yes I would have guessed that but I did not know that as a fact so in that way it is a fascinating fact yeah so back in 2019 I was like who the inventor of the PDF it's Adobe I never thought I I would be here but it subconsciously had it in my mind okay um

now exciting thing is that um I was just uh uh looking if if there is any opening okay because I'm more into document AI and then I'm only I I only selected and looked at okay these these These are the jobs that I should apply and then I just found one and then I applied one so that's the only thing I applied for Adobe so specifically I applied for that um and surprisingly I just got that um um um it was it was like that role was meant for me at the job description I read I felt like it was for me so that specifically I I had to reach out to the hiring manager for that specific role and then uh I straight away got the interview because even I think they felt the same uh connection so yeah it was very selective I guess that is so cool to me I mean that's honestly a flex right there where you can be like oh and

some you know there's people out there applying to I don't know like 2,000 jobs and you're just out here like oh I just applied to one and I interviewed and got that position so in that way that's really cool were the interviews fairly straightforward then presumably uh uh it's not straightforward mean I did apply to other companies as well but of course yeah I meant within Adobe yeah yeah within Adobe for the Adobe interview it was uh it was like an um series of interviews um so introductory round like hiring manager round then uh data structures and algorithms or uh take-home assessment round and then uh comes interviewing with uh team leads and then a uh principal machine learning engineer uh around where it is mostly on Tech um and then then then with the senior director and then uh um yeah then

that's it so it's like around five to six rounds yeah I see and so I'm curious to uh you know understand the type of technical questions you were asked but also maybe if you could use that opportunity to also talk about to the best of your knowledge what type of generally in the industry when you know companies hire for machine learning Engineers or you know AI engineers what what sort of technical questions can one expect in that yeah definitely first thing uh first thing is that uh because they have everyone has to be good with coding they would test the coding skills and it can be tested in either data structures and algorithms way or ML algorithms from scratch by that I mean like just Implement uh logistic regression from scratch or uh linear regression, K-means, SVM, and basic ML

algorithms or data structures or uh take home assessment so take-home assessment is something that the company is actually working on and uh how would you solve it how would you approach the problem like within one or two days um yeah so that's these are the three categories I would categorize as like for the coding part and the rest of the interviews mostly follow up on your uh previous work that you have done your resume your projects um and how good are you with ML breadth and how good are you with depth in ML so they can go into depth I try to ask how do how does this work how the Transformer work can you explain the architecture of Transformers um and uh things like that so there'll be like ML breadth ML depth and uh you have your research projects super helpful yeah super helpful were there

any resources that maybe you remember using that were helpful for you in in this preparation uh again for breadth it's always the basics ML depth uh I can share few it's like a it's not like a specific one place for everything uh definitely Coursera is one thing uh definitely helps uh because there are like lots and lots of courses on each of the um track in ML but there is no specific set of resources for everything at one place as of now makes sense that might be a problem worth solving maybe yeah I think I think it is actually um create something like create your own um I don't say GitHub repository or I don't say page but I would say something like um collection of good blogs uh is it's better yeah right where you can put research papers you can put the same place exactly and maybe just to you know as a very baby

step scratch at that what I could do is whatever resources that you can find if you can just send them my way I can link all of them in the you know show notes for this episode so that at least anybody listening that's trying to go down this route will at least have a starting point and from there maybe they will be finding other resources but I think that would be a great starting point so I'm going to bug you offline for that stuff if that's okay definitely I do have some resources and I would share like my prep strategy what I did use and then I can share few other things amazing that sounds great and yeah thanks very much for that continuing uh talk me through you know your you know maybe maybe you could contrast for us A Day in the Life maybe when you first started and you know maybe your day in the life yesterday

like what has changed What's the culture like what type of projects do you work on really just looking for context around you know what an engineer does or a machine learning engineer does that yeah so my typical work let's say uh would involve something like um It's a combination of research plus again application development I'm emphasizing this again uh because uh there's a lot to do with GenAI. You have a lot to ideate and explore so if I wanted to develop a chatbot I wanted to do something like prompt tuning what I would have to do I would have to kind of experiment check on multiple prompts do a lot of experimentation and then evaluate how good they are performing so this is an iterative process so I would generally uh code for some time and then attend some meetings and then code again for some time uh and then always kind of

document my work uh because that is very important I would say uh because whenever you are doing something um you'll always miss out um you just do it and then just close it but I would definitely keep a documented version of what I'm doing that day so that I can just go back and refer um so mostly my work will be on uh improving the existing system so right now as we are working on chatbot-like AI assistant for Adobe Acrobat so I'm trying to improve that uh uh in terms of uh responses in terms of suggestions in terms of summarization in terms of question answering things like that got it and so you've been working on uh Adobe Acrobat the whole time that you've been at Adobe is that right cool can you give us an you know maybe an example if you're willing to talk about or if you're allowed to talk about uh

some of the features that you've shipped maybe uh not too much I guess but on a high level yes uh so yeah that would be great yeah so I think it's mostly on the um um Adobe AI Assistant is pretty new uh it's it it came out in the last year uh around November I guess um to the public and um and so if you open Acrobat you have Acrobat Reader you have something called AI Assistant on a panel or on your right where uh when you open your document it provides your summary automatically and then um oh interesting got it yeah so imagine this for a student uh who's like a researcher or uh doing a master's or undergrad and then they have this bunch of research documents or research papers or imagine they have this ebooks uh and then they wanted to prepare or summarize or ask questions uh they can now do it easily

uh they don't need to use ChatGPT for that uh now they can directly on their systems itself they can directly ask questions get answers do follow-ups and then everything um uh remains secure and um so the whole uh question answering quality is what I'm working on how do you evaluate the quality of an answer how good is it uh can we build some sort of Auto evaluation metrics to understand how good your answer quality is is something I'm working on so it happens in every area uh every uh big company who's working on chatbots so it's it's fine to say oh yeah no that makes sense yeah and appreciate you sharing that I actually didn't even know that there was this feature so while you were sharing that I just opened it on my you know laptop here and yeah it it looks great I see it has some prompts

already um provide a list of five most important points what's this document about create an email summary all of these you know like yeah it does make sense that Adobe would do this because you know why have people go elsewhere when you can just do it in house so to speak um in terms of the applications itself I obviously I can understand that summarize or summary style applications are a big one other than that I'm I'm trying to think of what some other applications of this might be because because it's a PDF you probably wouldn't want to generate a lot of things or or is that not right I guess I'm just trying to think understand what other applications there are um other than the summarizer uh apart from summarization and question answering there are features like uh if you want to convert it to PowerPoint you can do that from

the PDF. Wow yeah I mean these are like well-known features so not everyone knows these features um um and uh you can you can highlight edit uh do a lot of stuff with the PDF so um now you can convert this to PowerPoint you can do um you can ask it to delete some sort of page um and it does it for you um yeah you can ask it to uh add some sort of image and you can directly in your PDF itself you can ask it to generate an image which uses Firefly uh to generate an image so let's say you can ask I want an image of rabbit and it creates an image can just drag it into your PDF uh on the web and it will edit the PDF and then I can just save it yeah you can edit a PDF I guess that's true yeah you get your image uh from the Firefly or creative side of

things and then you can put it in your uh PDF so imagine like you want uh it'll actually improve your productivity uh so and creativity too of course absolutely um something that I've struggled with a decent amount with Adobe is when you have to do the like sign stuff you know like you have a digital signature and I think that is that would be one which would be a great application for this where maybe you could just upload one sign and ask it to apply it everywhere in the document where it asks for Signature I don't know if this is being considered it's there I guess already like using generative AI uh why do you need a generative AI for that so I guess so imagine sometimes when you have a 60-page document and you don't want to go through and manually attach it everywhere maybe you just do it once and the thing does it for you yeah

just like how DocuSign does it right uh right but DocuSign makes you click though all 60 times I'm saying I don't want to do that I think legally that we need to I think we need to check each page like wherever signature is there that's the whole point of signing right that's true yeah I I think I put the you know cart ahead of the horse on that one so yeah that makes sense um I'm also curious to learn about how you integrate or maybe you don't do that personally but uh really the other aspects of AI using so obviously Photoshop is a big big Contender for something that could really be transformed by some really good good old generative AI so do you know people or do you ever talk with other Adobe folks about that type of stuff I did talk I do talk to people uh and then there are that side of uh things are really going crazy uh uh

the way they are competing and building things is different because it's it's a it's visual people can uh relate more uh get more um enthusiastic about it okay if I want to create a video out of image I can do it now if I wanted to create a a five-second clip or 10-second clip uh out of the text-to-video I can do that so these are some of the applications of the creative side of the things Photoshop and other side uh and imagine that if you want to edit a photo and you are very much a beginner you just started editing and you don't know how to edit your photos you just type okay I want to add uh I want to make it fair I want to remove the beard I want to add more hair I want to uh crop the picture I want to reduce the facial fat things like that you just type in and then it automatically does for you

now so these are again generative fills based on text so interesting use cases for the Creative Cloud as well so the reason I asked specifically was I have been so you know just for some context for my thumbnails sometimes I I try to make so for instance I you know recently recorded an episode with an artist so this person she is one of the best impressionist artists in Chicago and for the thumbnail for that I would have or ideally I would want to show both of us in the style of you know um like a medieval artist maybe mon or somebody and you know with like the beret and some like whatever pipe or something you get it I mean except I have no idea how to do that I don't have the props and I'm not manually going to go to a photo studio to shoot that so this is something that I have somewhat experimented with not with Adobe but

just with I think um DALL-E is what I've used and I found it to be very hit or miss so a lot of times it will generate what I wanted to but whatever it generates that person does not look like me despite me giving it you know four five really high-quality, well-lit pictures of my face so I find that it takes me a lot of iterations to get it to do even like something that I can pass off so yeah the reason I asked the reason I bring this up uh Nikhil is do you think like obviously this will improve but how often or how quickly do you think we will see where it's just really good like one prompt is all it needs to get you exactly what you want which I would argue ChatGPT is pretty good at that usually I find that on the first attempt it gives me what yeah um so all these models that you're working on DALL-E and everything it's text-to-image

um yeah there are there's a lot of improvement first thing I would agree on that and then second one with the with these upcoming video models uh I think we should be already at that stage um the next version that might be released by OpenAI uh for the DALL-E and any other model should should have this uh give it three to four images of yourself and then I want you uh to be in this style I want you to be in French style or some other style so it it is possible uh maybe whenever they release the next model even I'm waiting for that actually so uh it's just more uh training more feedback collecting more feedback and then uh iterating on top of it improving the existing models uh and also all the feedback that users provided within the last year is taken into consideration when when they're training the new models so that way they

know how to optimize it. What's the rationale behind preventing it from looking at feedback before a year just just outdated feedback uh the year before like you said they just look at feedback for the last year yeah so because the generation was there but uh generally how do these text-to-image models work is that they start with nothing um I don't want to go into technical details but uh they try to mimic what you're expecting with your uh uh with the compared image like with the output they had so they need to have initially they need to train something like what is the expected image what an expected image would be so but in your case uh you found them not good quality generation so then when you give feedback they're stored and then then they when they're training the next time like they also incorporate

these things uh into the training uh training as training data uh and also there is something called uh reinforcement learning in real time so even that helps that is what is helping you when you're typing something and then we you are typing this is this is I'm not satisfied with this image generate few few few more new images with this style so that is what is happening in the real time that's where you're seeing some some improvements when you're trying again and again and again interesting that makes a lot of sense how it it gets better the second third time you know it keeps getting closer and closer so yes that part makes sense um the the other thing that I was curious to get your take on was when you're working on you know as we just discussed it's such Cutting Edge it's literally we are on the brink of you

know the next big thing here you know or at least you are I'm I'm also on it but from the other side as a consumer yeah but when it comes to you know productizing these things I don't know if that's a word but yeah really just the verb form of making something for the masses a lot of those people that don't understand the nuances of these Technologies they don't know how it works they just wanted to do what they want what are some considerations that you obviously as an engineer that works or that builds this type of stuff what are some considerations that you look at so that you know it's not just about building the best thing that you can build but it must also come with the you know the UI/UX that must accompany any good product or must make any good product so I guess I'm trying to understand what this or how do you make

good UI/UX specifically for generative AI yeah uh that's actually a very big question and so why I would say that is um in big tech companies unlike startups big tech companies are huge they have whole bunch of teams different teams like product teams you have front-end, back-end you have uh UI/UX teams uh you have your research teams there are a lot of teams actually um so it's all a again a collaboration effort uh no one individual is can can talk about everything but it's again a collaboration uh whole whole part of collaboration so everyone every team does what they're good at so let's say if UI team has works on the different designs do some A/B testing with the real world users and then get their feedback which one is good and so similarly uh as uh as developers we ship product and then we do A/B testing as well what

features work best what features doesn't so it's always an iterative step we used to deploy collect the feedback and then based on uh what sort of what worked out best like 60% of users like the approach A 40% of users approach B so we're going to go with approach A and there are different mechanisms to do that so and also collecting the feedback uh from the users really helps uh because that's who we are building for so uh definitely it's the whole collaboration effort and then uh feedback is what it is got it so you're just getting you're informed by other teams that work more specifically around this type of stuff and then when you're doing the coding that's when you're trying to make sure that you incorporate all of that feedback that you received so that the end product is that you know well polished it contains

everything that the entire unit worked on not just yours specifically yes so as a bunch of teams small teams uh we work on individual features let's say So based on the user feedback where the Improvement is expected each feature each feature teams will actually try to improve uh their work over the time and as you go slowly towards you know each quarter you kind of improve your process so things move uh kind of slow but uh uh that is how it should be for the big tech for the large scale unlike startups fair enough yeah I mean if you're shipping directly to millions of people worldwide you would definitely want to make sure you have your house in order and you're not just shipping some half-baked or not well properly tested uh you know feature that then otherwise breaks people maybe their computers or their

brains and yeah that's just obviously PR disaster so I can appreciate that I suppose exactly yeah yeah that that's what I think yeah all the big Tech focuses on meta and everyone Amazon every company true true and um what do you think is the future of let's just start with Adobe Acrobat it's just something that again I've used my entire life doesn't seem to change much solid 9 out of 10 gets the job done and you exit and you're good right yeah what is the future here that you envision it doesn't have to be necessarily you know leaking secrets that you're aware of from Adobe but I'm just trying to get your personal take on where we're headed uh it's more about adding GenAI, like assistant is one thing search assistant these are some of the important components having good search on your PDFs and having good chatbots and

question answering reasoning platform is another thing and then having your generative AI images and editing that would be one thing and if you want to go into the the future and based on how you interacted with them what sort you can do some personalization on your own like uh uh think about user personalization uh so you are a doctor now uh and you have a bunch of documents related to doctor. Because this AI knows you are a doctor so it gives answers based on your level of uh depth that um so and think that is one example and then also you know about ChatGPT has its own memory right uh you know it right like it has memory so um so key points that you give to ChatGPT it stores in its memory so internal memory so having a long-term memory always helps

uh unless you delete that long-term memory so that can be one another thing good use case think about it's not just one document think about like 100 documents and you have your whole database or set of folder or set of directories in your local system or any Cloud platform which you wanted to search on which you want to question on so let's imagine there's 100 documents which you want to question on and you don't specify where the answer is within 100 documents but you get the answer wow that case is not for ChatGPT yeah and it it will never be right like ChatGPT will never probably I mean it shouldn't um have access to people's you know like local directories or their files so that's a really good good point I had actually not considered how that would really be clutch um and another thing that came to mind when you were talking

about the doctor example is you know say I'm a student uh preparing for an exam I have the PDF in front of me and maybe I can ask it to you know generate a question paper based off content over there and I can practice maybe math or whatever I'm studying and you know it can regulate the difficulty level cuz you start slow and then you can be like okay I did nine out of 10 make it harder I think that would also be such a cool you know use case for this similar use case you can generate draft um High School question paper itself like whole question paper you know uh or undergrad question papers uh the professors can do that like given a set of materials this is the level of difficulty or three out of five uh design a paper uh question paper with 10 questions uh where three of them are these three of them are like true or

false and then three of them are essay type of questions and it does it for you yeah and even for grading they could probably also use it I mean maybe they shouldn't but if if it were to come to that they could just upload somebody's term paper and we would probably have handwriting to text conversion at a really good spot if it is even handwritten and obviously if it's digital then we're good anyway and then it would just you know grade it for the you know teachers as well so yeah it really does sound like endless um Endless Possibilities here there are actually um applications that use ChatGPT to grade answers um I've seen some platforms do that yeah that's amazing like they they just like let's say there was there are

there's one essay written by someone and there's another essay written by someone else so based on the depth or let's say they score it like four out of five or 3.5 out of five it was like amazing yeah no you don't need to read everything you just pass it to GPT and then you get the answer do have you noticed or have you ever thought about some worrying like obviously this stuff is all so cool and flashy like there's no going around that but does the part of you ever wonder that maybe we've reached a point where or maybe in the future we might reach a point where some of this gets problematic in various ways in that you know students stop thinking and apply themselves teachers stop doing what they're supposed to do, which is grade papers and all that I guess are you is any part of you a doomsday person

when it comes to generative that's a good question uh um it's a mix mix of answer so I would say like um being able to use these tools is also a skill so fair yeah yeah but if you want to know how to use them properly you have to have the knowledge already or either should learn from it that way you know what to ask exactly so if you ask right questions you get the right answers garbage in is garbage out basically so you have to ask right questions to get right answers so in order to ask right questions you need to have some sort of knowledge um you can't bring some uh uh some person from other domain and ask him to uh uh ask questions on you know uh film making or something like that because they don't know much about it and it will be really generic like you know their insights and it's it will be very surface level that

makes sense yeah guess and prompt engineering and these are other things everyone should learn eventually the next two years so that they can ask better questions better ways of asking questions uh to get better answers um yeah but doomsday is is something um because it doesn't kill your creativity it actually improves the way you consume your knowledge uh and it it helps you consume knowledge quickly so that in in one one hour if you're learning x amount of knowledge now you can actually use that to learn 3x or 4x if you see it in the right way if you see it in another way it it is bad so there's a mix uh no that's I I get what you're saying the the thing that trips me personally again this is just very subjective um and I've actually done this experiment where you know like just find any book chapter

whatever five pages spend time reading those five pages that's you know a and then B is just copying that entire all of those words and pasting it into any LLM ask it to summarize I find personally that the type of insights that my brain generates for me when I'm reading something you know word to word the old-fashioned way it's funny we have to say that now is very different from what I get from reading that summary like sure it does a good job obviously it is a summary but I find that for me to really absorb information at a deep level it I find it maybe it's just placebo I don't know but it does lack somehow so yeah have you had any such experiences yeah let's discuss the same example like how I would do because this seems like interesting example um so you you might have just asked okay summarize this book

uh based on each chapter yeah or just that chapter like just summarize this chapter, a single chapter yeah so what I would ask is um include all the points where there is like interesting stories think from a human perspective and uh list out each and every point that you feel like this is impactful and then connect it to the examples provided in each chapter and if there is any sort of story that I have to know or any sort of example that I wanted to know uh you have to Output these examples or you have to also give these examples don't miss out on the storyline give the storyline in a sequence this is something I would give as a prompt so detailed set of instructions then what it does is it will give each and every point that is so important in an order so when you read you feel like okay you have all the context and let's say you

had an example of okay uh this person did this so and so did this this person did this so and so and so in the book if you try to summarize it doesn't give that example out right exactly so now if you ask your real use cases; give me use cases uh and don't skip any topic per page don't skip any you just have to prompt it right uh and that way you can get now that LLMs have so much of a context to fit in they might actually give around half of what is important from that chapter yeah probably more honestly the way you describe that prompt I can see it doing a really good job because I use it fairly often so I I can now by now I'm I'm at a place where I can correlate the type of responses a potential prompt would get I think your prompt would kill like I think it would do a great job

Prompt engineering takes you to a different stage actually that's why they pay like 300K for prompt engineers out there oh I had no idea wow that's that seems wrong is that right I don't know if they're paying now but there was a time in the last year where they said that they were paying like 300K for prompt engineers and everyone was so surprised now now that there are websites and there are tools that generate prompts for you if you give uh prompt and ask how do you want it now there are tools that expand your prompt, rewrites your prompt uh rephrases it like that yeah probably there's a few courses out there as well I'm sure that people could take to get really good at prompt engineering as well like they

have to be at this point yeah no matter where they're coming from what time and if they are using ChatGPT, this is a must learn yeah I really like that actually I feel like that what you just shared really has reshaped pretty much every time I'm going to use ChatGPT I'm going to keep that in mind that I need to help it help me like just doing a you know half-baked job like you know just do XYZ is not good enough if I go that extra mile write those two more sentences that will massively impact what I get out of the LLM in return so I think that's that's something that has stuck with me I feel like and it will I will be taking that you know going forward and I'm sure our listeners will as well so really appreciate that such a nice Insight that you just shared amazing yeah thanks yeah of course so those are really all the questions I had

before we close out here would you is there anything you would like to share with with our listeners um I mean uh ML is huge. If you're rushing and trying to get a job in GenAI uh just focus on the basics of both ML and GenAI so that and build projects and because it's so underrated people don't build; people just do courses I would suggest if you really want to get into the GenAI side of things build projects learn learn from the applications and it takes you a long way over the years Absolutely, Nikhil. Thank you so much for taking the time today it's been such a blast to talk with you not just about your journey but really just um you know brainstorming I think it's the probably the right word uh about generative AI applications and maybe you know some of my ideas were really bad but it was still really fun to bounce them off you and you know

really just pick your brain about this stuff because it does obviously I'm sure that you know this stuff so well and it really just shows so thank you so so much yeah it's really fun talking to you Naman So yeah thank you so much that brings us to the end of episode 35 of the Ready Set Do podcast thank you all for sharing these episodes with those that continue to benefit from them if you would like to support me the easiest way to do that is by subscribing to my YouTube channel leaving me a five-star rating on Spotify or any other podcast app of your choice or checking out the links in description for additional ways to support me catch you all in the next one new episodes every Wednesday