How To Design Generative AI Features For Adobe Acrobat (& Break Into Machine Learning Engineering Roles) - w/ Nikhil

if I want to forge a career in machine learning and or artificial intelligence what should I focus on should I look at building stuff or should I upskill myself doing lead codes so firstly it's not just one part what did chat GPT do that made them the biggest name in Tech last year machine learning is almost everything is about research some of my friends they don't come from traditional research backgrounds and neither do they have any inclination to do any research are there any courses that would be good for people that are passionate about machine learning I had to reach out to the hiring manager for that role and then I straight away got the interview I mean that's honestly a flex right there how do you make good uiux specifically

how do you make good uiux specifically for generative a right now we are working on chatbot like AI assistant for the Adobe Acrobat what was your interview experience like how did you end up at Adobe there was a time last year where they were paying like 300K for prompt in days and everyone was so surprised Welcome to The Ready Set do podcast where we learn from journeys of not experts who are just two steps ahead of us I'm Naman Pand and in this episode featured not expert is nikil pental nikil is a senior machine learning engineer at Adobe currently working on building Cutting Edge generative AI features on Adobe Acrobat throughout our discussion nikel references his experiences and learnings from working on document intelligence for almost 5 years now to walk us through the best

years now to walk us through the best ways for current students and professionals to get hired for their first machine learning engineer role I think you espcially enjoy our Deep dive on the best strategies to set oneself apart by building machine learning projects and how that ties into the interview process for most machine learning engineering roles Nel also takes us through a day in the life at Adobe and what the future holds for the generative AI capabilities of Adobe Acrobat which is the most popular PDF tool worldwide I also thought Nichol's take on the impact of AI in the not so distant future was particularly interesting in keeping with our theme of learning from somebody that's just two steps ahead of us instead of an expert

steps ahead of us instead of an expert my goal with this episode is to highlight Nichol's Incredible Journey and how he was able to forge a career in this niche of document int this is The Ready Set to podcast and to support it please subscribe to the YouTube channel and or leave me up to a f star rating on Spotify or any other podcast app of your choice you can also check out the links in description for other ways to support me some being more direct than others and now without any further Ado my friends here's Nik nikil welcome hi Naman it's good to talk to you let me begin with the question that's really on well I was going to say everybody's mind but seeing as that

everybody's mind but seeing as that there's nobody else here just my mind um and the question is if if I'm a current engineer or I'm a current student say studying computer science maybe btech or Masters maybe it doesn't matter and if I want to you know Forge my way into a career in machine learning and or artificial artificial intelligence should I you know what should I focus on I guess is the question should I look at more projects and that type of you know building stuff or should I upskill myself in terms of learning new technologies doing lead code so which approach would you prefer or is it a mix of both of them yeah um I can talk on that so uh firstly it's not

can talk on that so uh firstly it's not just one path it would always be a mix you have to attack the problem from all the directions so uh so what I mean by that is uh you have to have good coding skills because and not just coding skills you should should also have uh subject matter expertise where you get to learn about more about ml deep learning and um uh you have to do multiple courses courses just spill some theoretical knowledge right so but theoretical knowledge doesn't take you waste so I would say that uh apart from courses you also have to build something using the knowledge that you have learned from the courses and that way you'll also improve your software

you'll also improve your software engineering skills integration skills application skills and then overall ml knowledge in in action so from what you're saying it sounds like if somebody were to just focus on algorithms and data structures that would give them the base to then later go and start attacking machine learning specific topics uh you know specifically is that kind of what you're saying yes I mean data structures and algorithms are important so as a part of machine learning engineer he has to be good software engineer as well so okay one has to be a good coder uh uh uh it's it's it's related it's very related so uh if you have good software engineering skills that'll be a base to start for the coding part but rather then you can uh learn about ml algorithms and things

uh learn about ml algorithms and things like that and then it it'll become easy when you are building something makes sense that is what I thought um except then I was introduced to this notion at least I don't know anybody personally that does this but I know there are people online that talk about you know just really just hacking they're like screw all of this stuff who will learn uh you know all of these algorithm stuff let me just Cobble together a project you know like just hack together from taken from multiple places you know a workable prototype or a working prototype of anything really that solves a problem you could have one that generates recommendations whatever it's not very hard to build and and I would argue that you don't really need a lot of um you know coding skills to do that

of um you know coding skills to do that you need some but you probably don't need to understand how like ta algorithm works if you're trying to just Cobble together a project do you think that's still a sustainable way to learn machine learning or would you like not agree with that um I mean that's actually the best way because if you're building something right if you're building something and which means that you're learning it from the other side uh so not in the traditional way where you learn theory and then Implement uh if you're building something and learning hacking the way around building an application you will obviously have to learn something while you're building something so which is enough uh for you to build but now uh even though you feel like you hacked it around internally you

like you hacked it around internally you you you will learn some things not everything but at least few things how do you connect things how do you uh do things for the rest of the things you can definitely use chat gbd or any sort of large language uh uh interfaces you know to um to get it like because everything is available for free now um and there's you can always hack your way around but again let's say that you you want to build something from ground up there's nothing you can refer to some new application uh that is when uh I think Basics will help uh so it's it's again a trade-off so what you are trying to build is it is is some reusable components already exist in the market that I can use open source things that I can use or is it something that i' have

can use or is it something that i' have to build from the scratch so it's completely on the use case I see so what you're laying out is it's fine to hack you know hack through your way and build something but eventually if you do enough of that you will eventually reach a point where you will need to go back to basics and learn the foundation without which you will not be able to go that far like just building one project isn't most people's goals you like you would they would want to continue building and obviously eventually maybe get employed somewhere and for that reason or for if that is the goal they would probably benefit from having that overall knowledge not just the

overall knowledge not just the theoretical part not just the hacking part but some mixture of both of them will probably be most beneficial would you agree yeah so one more thing I wanted to add is um uh so I live in SF and then there are a lot of hackathons happen here and then I see a lot of students lot of uh working people come there and then uh they don't have a specific a domain knowledge there they might come from data analysis background they might come from software engineer background but the hackathons are related to let's say ml uh they can't just learn it and then do it uh on the same day so it's like a one day hackathon so now this is where the hacking Parts part starts so then uh as

hacking Parts part starts so then uh as like one individual will go uh find the references and then connect different things to make it work by the end of the day so I would say it's a good way to get your product product or prototype done to show to someone to show to some uh investors or anyone but uh again end of the day it's it's just a prototype MH makes sense and I guess seg to kind of the question I had for you so you personally when you were coming up um which approach did you use of the two that we discussed and maybe you can use that as a a way to also talk about you

that as a a way to also talk about you know your career trajectory and your career path that has seen you come to Adobe at this point where you were currently yeah so I started uh um when I was in bachelor's back in undergrad um uh back in us uh back in India so that is when I started my um uh data science Journey so mine was more in a traditional way in the initial days um but yeah let let me let me walk you through uh what happened um so after my undergrad I uh had some of the projects in ml back then uh sorry and was your undergrad just in computer science or was it more no uh so my undergrad was in

was it more no uh so my undergrad was in electronics and communication surprisingly I was told that uh if you take electronics and communication you can go either way it's true I did hear that also yeah but surprisingly it didn't work out that way but what happened was I developed an interest in ml because my major project in my undergrad was U the ML and then that is what led me uh to uh get placed in a company which and to a team where I actually am uh started as a data scientist SL machine learning engineer wow that's incredible so for me I would say the starting point was the uh major project that I've done um back in 2017 20 2017 2018 uh so that is when 2018 is when I

2018 uh so that is when 2018 is when I started my first job in as a data scientist I see and where was that as where were you working so I was working with Reliance back then uh Reliance Industries um as a machine learning engineer uh so yeah that's I'm an amazing team and the uh that is when the machine learning started to Boom actually uh the roles of data scientist and machine learning uh came and then it was rapidly increasing got it and are you willing to talk about you know just a day in the life what type of projects you were working on like what was going on maybe was it a tool or product that I would know or you know one of some of our listeners would be aware of yeah so

our listeners would be aware of yeah so back in the day um U so I work mostly in a natural language processing and computer vision just to go into depth kind of projects that worked on is mostly on the automating um um uh kind of a lot of document stuff and I worked extensively on document AI site so kind of uh automating extracting some of the important entities doing some classification building some classification models building some recommendation models and building an an mlops pipeline uh for these models so this would be like a traditional uh work that I've done back then um so ml Ops was get just getting started that time uh so it's all completely new there were no huge differences to how do you build

no huge differences to how do you build these pipelines how do you maintain we had to come up with our own approaches the tools were actually in early stages so it was it was amazing I I felt like I worked in a startup rather than in a big company that's so interesting I didn't even realize Reliance does a lot of document stuff so that's definitely news to me there are a bunch of teams um uh so and relance is not small right like Lance has associated with multiple small startups vendors everyone so all the businesses that they have everyone is going digital like uh so even the archives they had uh so they had to go digital and everything uh so everything

digital and everything uh so everything is now digitalized so apart from the document AI I also worked on something similar uh creating your own um photo platforms like similar to like Google photos uh but it's kind of internal to Reliance so similar to that I built something where the features are very interesting like okay face recognition you know search captioning um clustering of faces uh search over the images these are some of the interesting things done there that sounds amazing yeah and definitely I don't think it would come as a surprise or sorry I mean it would come as a surprise I think for a lot of our listeners that this these type of projects are you know were worked on uh in Reliance at the time and now I can only imagine what type of projects must

only imagine what type of projects must go on there because they've become even bigger and even better right right I think the lot going should be I think I should there should be a lot going on J side of things now absolutely yeah it does sound like the you know buzzword of our times for sure so um what did you do next then I I presume you went to the US for your Masters what was that in and what was that experience like yeah so I worked with Reliance for two and a half years and then switched to another company I worked there for a short time but then I got a chance to move to us uh it's through Masters I did

move to us uh it's through Masters I did my masters in computer science uh from University at Buffalo New York um so it was like a one and a half year course so uh pretty short so the main goal here was to just transition to us get a good job in the US so that's the whole idea and also if possible getting to good research University for the computer science so machine learning is almost everything is about research um so if you have a good research profile you get the job easily uh in in ml so that's one thing I knew before uh so I always wanted to get into good good uh University where there's like a balance between research and cost makes sense um

between research and cost makes sense um some of my friends have you know they've expressed interest in getting master's degrees from the US except they don't come from traditional research backgrounds and neither do they have any inclination to do any research do you think there is a market for such people or maybe there are there any courses that you know of that would be good for people that you know like this that are passionate about machine learning but are just not passionate about doing research or you know publishing papers yeah so I can give my example so I I did research like I've done so ml is more about anyway research research new techniques experiment experiment experiment till you get uh sustainable results and then uh you take it Forward uh so we come up with lot we

it Forward uh so we come up with lot we used to come up with lot of new techniques but no one in India encouraged us to publish the papers in the team so uh that was one drawback um I would if I would have to go back why was that do you know uh it's because the emphasis was just to kind of uh deliver the results deliver the code deliver the application to the users and it was never mostly on okay we as a team can can publish the paper uh that would help but because publishing also there are separate research teams actually uh so that does the research work but as part of uh generally in s in other few companies uh people do the research and people also deploy the applications to

people also deploy the applications to develop applications so they do end to end so but I never got a chance to publish a paper when I was with Reliance or any other company before um so that way I didn't had that research background uh our Publications but I I had to take a different approach I made sure that I built good projects so I had a good portfolio that will eventually help so if I had to suggest I would definitely suggest building good projects if you are not into the research side of things build good applications with some business outcome business value so that you can present yourself to some uh to recruiters or anyone um because there are two again two phases one on the research as I told

two phases one on the research as I told you and one on the application development and deployment s so you can always apply to ml uh engineer or ml Ops engineer or uh uh uh forward deployment engineer things like this so these are mostly towards the uh application development side that's really helpful context um yeah I can I'm happy to have covered that part because it is indeed a question that comes up often um yeah so then moving on uh what did you do next um I know you were you were at you know doing a co-op at a startup what was that experience like and what were you working on at the time yeah so um with

working on at the time yeah so um with the coop um I actually got an internship and then that got converted to co-op so internship I can do full-time like 40 hours a week but Co-op is like just 20 hours a week is something I could do so the main Pro problem statement that I worked there was mostly on uh again document classification and building the entity extraction and question answering pipeline so think about it like building a chat bot uh for your use case um so there are a bunch of insurance documents or or legal documents that I had um and I want to answer a question uh user might ask any question and then uh it has to get answered and then you need to

has to get answered and then you need to uh show them where the answer is coming from from a bunch of documents that they upload so this is a simple use case of a chat B right uh but the problem was that the data was so huge and then this domain was pretty new insurance domain uh so there were no much not much innovation in the insurance domain but that is where I I thought okay this uh startup was based on insurance so so that is why I was like okay let's try exploring this this part and see how it goes got it so how did that go it was fine uh it was fine I was able to uh

fine uh it was fine I was able to uh deliver uh better but um one it was very early stages back uh last year uh because chat GB just got released and um um and everything whoever is developing chat BS or whoever is developing llm based applications skyrocketed like crazy uh so and everyone started the Gen whole J I think started to boo J was there before also uh even before this CH gbd uh but yes gen is generative AI which means you generate something like token or create an image or things like that so we have but the conversational you know like wrapping around it I guess that was that there as well I'm genuinely curious I'm not trying to you know like doubt you so no it's there from 20 2018 2017 2018 so when things started like Transformers uh bird based

started like Transformers uh bird based Transformers and then other um uh Technologies came in other models came in so that is these are all language models are all generative models so uh so you try to just predict the next token basically so in your text so that is what it is so they were there from long time but it's just that infra is not there up to the mark and then the model training data is not up to the mark and um and lot of other things but now with charb it's it's so quick that you can develop realtime applications uh and you don't need to wait for that for the answer like a lot of yeah that's so interesting I guess the the part that is

interesting I guess the the part that is still not clear to me is what exactly did chat GPT do that nobody else had done before them because you mentioned if generative AI was still around everything was around so were they just credited with the infrastructure did they just happen to get lucky like I guess I'm trying to ask what did chat GPT do that made them you know the biggest name in Tech last year yeah I I would talk as a generic point of view uh because I am not from open AI I can't talk no but you're way closer to that than I will ever be so I'm just curious to get your thoughts definitely so first thing that happened over the time was uh

thing that happened over the time was uh shift with the infrastructure the gpus the compute um you know how Nvidia came up with a lot of gpus and then computer power and then uh parall open a was working uh on tra getting Gathering huge data sets uh open source all over the world um and then collecting more and more data training huge models so earlier in initial days the bird and everything they had like parameters every model has few parameters right uh while training so this these are called uh model parameters or uh so they have like 2 billion 10 10 billion to 100 billion things like that so as number of parameters in increases the model size increases now let's imagine back in 2018 if someone wants to develop a model with uh uh let's say uh 3 billion or two

uh uh let's say uh 3 billion or two billion parameters it's going to be so comput expensive uh because of the training process because of the hosting process because of the serving process what I mean by serving is if some user asks uh something it has the model has to serve like give answers and that's not that's not that wasn't possible in real time back then because of because there were no good gpus back then so as the time progressed regarding to Mora if you are aware of that um uh so as gpus Pro increased the compute power increased in 2021 2022 is when uh Nvidia booed as well in terms of GPU uh provisions and that is when opena uh also trained their models over the Nvidia gpus a00 and then they deployed

Nvidia gpus a00 and then they deployed them on on huge clusters huge clusters of gpus serving real time so that is why when you type a question you know in a chat gbd you get the tokens instantly just like that that's so interesting that it was you know not just them being at the right place in the right time but really sounds like so many external factors that just happened to all come together at the right time for them to you know deliver this really just sensation sensation I feel like in my time at least or in our times I don't think we have witnessed or maybe I can I should just speak for myself but you know since the

speak for myself but you know since the iPhone it it this really feels or smartphone I should say um this really just feels like the next Frontier of the you know big big like landmark tech Innovation would you agree I'm curious to hear your take on that I would definitely agree on it uh because um smartphone is one thing and then now um we have I I I I assumed this point uh we will reach this point in like 10 years in 2018 but it was just cut in half like in five years we were already there um so the amount of speed uh um with which we reached here is amazing I would say and Chad gbt is a game changer and OB is a game changer actually truly

a game changer actually truly indeed um yeah I guess continuing where you know sorry for that cite track uh but at least for me that was really really helpful because really like this whole time I had no idea how or why you know what you went over had happened so that's really helpful context but from what it sounds like through your career trajectory so far it feels like a no-brainer that you work at Adobe I know Adobe does stuff other than document stuff but because you know what you from what you laid out everything you've done is has been around document classification NLP like machine learning X documents essentially is the vibe that I'm getting so how did that come to be though I don't want to assume a lot so

though I don't want to assume a lot so what was the interview experience like what was your like how did you end up at Adobe essentially um yeah I was actually fascinated by a facts because I was working too much in document AI um I was always curious like who was the inventor of PDF and it turns out it is adob okay I yes I would have guessed that but I did not know that as a fact so in that way it is a fascinating fact yeah so back in 2019 I was like who's the inventor of PDF it's Adobe I never thought I I would be here but it subconsciously had it in my mind okay um now exciting thing is that um I was just

now exciting thing is that um I was just uh uh looking if if there is any opening okay because I'm more into document AI and then I'm only I only I selected looked at okay these these These are the jobs that I should apply and then I just found one and then I applied one so that's the only thing I applied for the adobi is so specifically I applied for that um and surprisingly I just got that um um um it was it was like that role was meant for me at the job description I read I felt like it was for me so that specifically I I had to reach out to the hiring manager for that specific role and then uh I straight away got the

and then uh I straight away got the interview because even I think they felt the same uh connection so yeah it was very selective I guess that is so cool to me I mean that's honestly a flex right there where you can be like oh and some you know there's people out there applying to I don't know like 2,000 jobs and you're just out here like oh I just applied to one and I interviewed and got that position so in that way that's really cool were the interviews fairly straightforward then presumably uh uh it's not straightforward mean I did apply to other companies as well but of course yeah I meant within Adobe yeah yeah within Adobe for the adobe's interview it was uh it was like an um series of interviews um so introductory

series of interviews um so introductory round like hiring manager round then uh data structures algorithms or uh take-home assessment round and then uh comes interviewing with uh team leads and then a uh principal machine learning engineer uh around where it is mostly on Tech um and then then then with the senior director and then uh um yeah then that's it so it's like around five to six rounds yeah I see and so I'm curious to uh you know understand the type of technical questions you were asked but also maybe if you could use that opportunity to also talk about to the best of your knowledge what type of generally in the industry when you know companies hire for machine learning Engineers or you know AI engers what what sort of technical questions can one expect in that yeah definitely first

expect in that yeah definitely first thing uh first thing is that uh because they have everyone has to be good with the c good with coding they would test the coding skills and it can be tested in either data structures algorithms way or ml algorithms from scratch by that I mean like just Implement uh logistic regression from scratch or uh linear regression K means SC and like basic ml algorithms or data structures or uh take home assessment so take-home assessment is something that the company is actually working on and uh how would you solve it how would you approach the problem like within one or two days um yeah so that's these are the three categories I would categorize as like for the coding part and the rest of the interviews mostly follow up on on your

interviews mostly follow up on on your uh previous work that you have done your resume your projects um and how good are you with ML bread and how good are you with depth in ml so they can go into depth I try to ask how do how does this work how the Transformer work can you explain the architecture of Transformers um and uh things like that so there'll be like ml breadth ml depth and uh you have your R projects around super helpful yeah super helpful were there any resources that maybe you remember that you use that were helpful for you in in this preparation uh again for bread it's always the basics ml depth uh I can share few it's like a it's not like a

share few it's like a it's not like a specific one place for everything uh definitely corera is one thing uh definitely helps uh because there are like lots of lots and lots of courses on each of the um track in ml but there is no specific set of resources for everything at one place as of now makes sense that might be a problem worth solving maybe yeah I think I think it is actually um create something like create your own um I don't say GitHub repository or I don't say page but I would say something like um collection of good blogs uh is it's better yeah right where you can put research papers you can put the same place exactly and maybe just to you know as a very baby

maybe just to you know as a very baby step scratch at that what I could do is whatever res sources that you can find if you can just send them my way I can link all of them in the you know show notes for this episode so that at least anybody listening that's trying to go down this route will at least have a starting point and from there maybe they will be finding other resources but I think that would be a great starting of point so I'm going to bug you offline for that stuff if that's okay definitely I do have some resources and I would share like my trip strategy what I did use and then I can share few other things amazing that sounds great and yeah thanks very much for that continuing uh talk me through you know

continuing uh talk me through you know your you know maybe maybe you could contrast for us A Day in the Life maybe when you first started and you know maybe your day in the life yesterday like what has changed What's the culture like what type of projects do you work on really just looking for context around you know what an engineer does or a machine learning engineer does that yeah so my typical work let's say uh would involve something like um It's a combination of research plus again application development I'm emphasizing this again uh because uh there's lot to with the Gen you have lot to um id8 explore so if I wanted to develop it some chatboard I wanted to do something like prom tuning what I would have to do

like prom tuning what I would have to do I would have to kind of experiment check on multiple prompts do a lot of experimentation and then evaluate how good they are performing so this is an iterative process so I would generally uh code for some time and then attend some meetings and then code again for some time uh and then always kind of document my work uh because that is very important I would say uh because whenever you are doing something um you'll always miss out um you just do it and then just close it but I would definitely keep a documented version of what I'm doing that day so that I can just go back and refer um so mostly my work will be on uh improving the existing system so right

improving the existing system so right now as we are working on chatbot like AI assistant for the Adobe Acrobat so I'm trying to improve that uh uh in terms of uh responses in terms of suggestions in terms of summarization in terms of question answering things like that got it and so you've been working on uh Adobe Acrobat this the whole time that you've been at Adobe is that right cool can you give us an you know maybe an example if you're willing to talk about or if you're allowed to talk about uh some of the features that you've shipped maybe uh not too much I guess but on a high level yes uh so yeah that would be

high level yes uh so yeah that would be great yeah so I think it's mostly on the um um Adobe a assistant is pretty new uh it's it it got got out in the last year uh around November I guess um to the public and um and so if you open your acrobat you have Acrobat Reader you have something called a assistant on a panel or on your right where uh when you open your document it provides your summary automatically and then um oh interesting got it yeah so imagine this for a student uh who's like a researcher or uh doing a Masters or undergrad and then they have this bunch of research documents or research papers or imagine

documents or research papers or imagine they have this ebooks uh and then they wanted to prepare or summarize or ask questions uh they can now do it easily uh they don't need to use chat GB for that uh now they can directly on their systems itself they can directly ask questions get answers do follow-ups and then everything um uh remains secure and um so the whole uh question answering quality is what I'm working on how do you evaluate the quality of an answer how good is it uh can we build some sort of Auto evaluation metrics to understand how good your answer quality is is something I'm working on so it happens in everywhere every AR uh every uh big company who's working on chatbots so it's it's fine to say oh yeah no that

it's it's fine to say oh yeah no that makes sense yeah and appreciate you sharing that I actually didn't even know that there was this feature so while you were sharing that I just opened it on my you know laptop here and yeah it it looks great I see it has some prompts already um provide a list of five most important points what's this document about create an email summary all of these you know like yeah it does make sense that adobe would do this because you know why have people go elsewhere when you can just do it in house so to speak um in terms of the applications itself I obviously I can understand that summarize or summary style applications are a big one other than that I'm I'm trying to think of what some other

trying to think of what some other applications of this might be because because it's a PDF you probably wouldn't want to generate a lot of things or or is that not right I guess I'm just trying to think understand what other appc there are um other than the summarizer uh apart from summarization and question answering there are features like uh if you want to convert it to PowerPoint you can do that from the P wow yeah I mean these are like well well known Fe not not well-known features so not everyone knows these features um um and uh you can you can highlight edit uh do a lot of stuff with the PDF so um now you can convert this

the PDF so um now you can convert this to PowerPoint you can do um you can ask it to delete some sort of page um and it does it for you um yeah you can ask it to uh add s some sort of image and you can directly in your PDF itself you can ask it to generate an image which uses Firefly uh to generate an image so let's say you can ask I want an image of Rabbid and it creates an image can just drag it into your PDF uh on the w and it will like edit the PDF and then I can just save it yeah you can edit you can

just save it yeah you can edit you can always in you can edit a PDF I guess that's true yeah you get your image uh from the Firefly or creative side of things and then you can put it in your uh PDF so imagine like you want uh it'll actually improves your productivity uh so and creativity too of course absolutely um something that I've struggled with a decent amount with Adobe is when you have to do the like sign stuff you know like your have digital signature and I think that is that would be one which would be a great application for this where maybe you could just upload one sign and ask it to

could just upload one sign and ask it to apply it everywhere in the document where it asks for Signature I don't know if this is being considered it's there I guess already like using generative AI uh why do you need a generative AI for that so I guess so imagine sometimes when you have a 60-page document and you don't want to go through and manually attach it everywhere maybe you just it once and the thing does it for you yeah just like how DocuSign does it right uh right but DocuSign makes you click though all 60 times I'm saying I don't want to do that I think legally that we need need to I think we need to check each page like wherever signature is there that's

like wherever signature is there that's the whole point of signing right that's true yeah I I think I put the you know cart ahead of the horse on that one so yeah that makes sense um I'm also curious to learn about how you integrate or maybe you don't do that personally but uh really the other aspects of AI using so obviously photosop is a big big Contender for something that could really be transformed by some really good good old generative AI so do you know people or do you ever talk with other Adobe folks about that type of stuff I did talk I do talk to people uh and then there are that side of uh things are really going crazy uh uh the way they are competing the they are building things is different because

building things is different because it's it's a it's visual people can uh relate more uh get more um enthusiastic about it okay if I want to create a video out of image I can do it now if I wanted to create a a 5sec clip or 10 seconds clip uh out of the text text to video I can do that so these are some of the applications of the creative side of the things Photoshop and other side uh and imagine that if you want to edit a photo and you are very big uh beginner you just started editing and you don't know how to edit your photos you just type okay I want to add uh I want to

type okay I want to add uh I want to make fair I want to remove the beard I want to add more hair I want to uh crop the picture I want to reduce the facial fat things like that you just type in and then it automatically does for you now so these are again generative fills based on text so interesting use cases for the Creative Cloud as well so the reason I asked specifically was I have been so you know just for some context for my thumbnails sometimes I I try to make so for instance I you know recently recorded an episode with an artist so this person she is one of the best impressionist artists in Chicago and for the thumbnail for that I would have or ideally I would want to show both of us

ideally I would want to show both of us in the style of you know um like a medieval artist maybe mon or somebody and you know with like the ber and some like whatever pipe or something you get HT I mean except I have no idea how to do that I don't have the props and I'm not manually going to go to a photo studio to shoot that so this is something that I have somewhat experimented with not with Adobe but just with I think um di is is what I've used and I found it to be very hit or miss so a lot of times it will generate what I wanted to but whatever it

what I wanted to but whatever it generates that person does not look like me despite me giving it you know four five really high quality well lit pictures of my face so I find that it takes me a lot of iterations to get it to do even like something that I can pass off so yeah the reason I asked the reason I bring this up uh nikil is do you think like obviously this will improve but how often or how quickly do you think we will see where it's just really good like one prompt is all it needs to get you exactly what you want which I would argue chat gbt is pretty good at that usually I find that on the first attempt it gives me what yeah um

first attempt it gives me what yeah um so all these models that you working on Del and everything it's text to um image um yeah there are there's a lot of improvement first thing I would agree on that and then second one with the with these upcoming video models uh I think we should be already at that stage um the next version that might be released by open AI uh for the Del any and any other model should should have this uh give me like three to four images of yourself and then I want you uh to be in this style I want to be you to be in French style or some other style so it it is possible uh maybe whenever they release the next model even I'm waiting for that actually so uh it's just more

for that actually so uh it's just more uh training more feedback collecting more feedback and then uh reating on top of it improving the existing models uh and also all the feedback that users provided within the last year is taken into consideration when when they're trading the new models so that way they know how to optimize H is what's the rational behind preventing it from looking at feedback before a year just just outdated feedback uh the year before like you said they just look at feedback for the last year yeah so because the generation was there but uh generally how does these two images work is that they start with nothing um I don't want to go into technical details but uh they try to mimic what you're expecting with your uh uh with the compared image like with an output

the compared image like with an output that they had so they need to have initially they need to train something like what is the expected image what would an expected image so but in your case uh you found them not good quality generation so then when you give feedback they're stored and then then they when they're training the next time like they also incorporate these things uh into the training uh training as training data uh and also there is something called uh reinforcement learning uh in the real time so even that helps that is what is helping you when you're typing something and then we you are typing this is this is I'm not satisfied with this image generate few few few more new images with this style so that is what is

with this style so that is what is happening in the real time that's where you're seeing some some improvements when you're trying again and again and again interesting that makes a lot of sense how it it gets better the second third time you know it keeps getting closer and closer so yes that part makes sense um the the other thing that I was curious to get your take on was when you're working on you know as we just discussed it's such Cutting Edge it's literally we are on the brink of you know the next big thing here you know or at least you are I'm I'm also on it but from the other side as a consumer yeah but when it comes to you know productizing these things I don't know if that's a word but yeah really just

if that's a word but yeah really just the verb form of making something for the masses a lot of those people that don't understand the nuances of these Technologies they don't know how it works they just wanted to do what they want what are some considerations that you obviously as an engineer that works or that builds this type of stuff what are some considerations that you look at so that you know it's not just about building the best thing that you can build but it must also come with the you know the uiux that must accompany any good product or must make any good product so I guess I'm trying to understand what this or how do you make good uiux specifically for generative AI yeah uh that's actually a very big question and so why I would say that is

question and so why I would say that is um in big tech companies unlike startups big tech companies are huge they have whole bunch of teams different teams like product teams you have front end back end you have uh uiux teams uh you have your research teams there are a lot of teams actually um so it's all a again a collaboration effect uh no one individual is can can talk about everything but it's again a collaboration uh whole whole part of collaboration so everyone every team does what they're good at so let's say if UI team has works on the different designs do some AB testing with the real world users and then get their feedback which one is good and so similarly uh as uh as developers we ship product and

uh as developers we ship product and then we do AB testing as well what features work best what features doesn't so it's always an iterative step we used to deploy collect the feedback and then based on uh what sort of what worked out best like 60% user like the approach a 40% users approach B so we're going to go with approach a and there are different mechanisms to do that so and also collecting the feedback uh from the users really helps uh because that's who we are building for so uh definitely it's the whole collaboration effort and then uh getback is what it is got it so you're just getting you're informed by other teams that work more specifically around this type of stuff and then when you're doing the coding that's when

you're doing the coding that's when you're trying to make sure that you incorporate all of that feedback that you received so that the end product is that you know well polished it contains everything that the entire unit worked on not just your specifically yes so as a bunch of teams small teams uh we work on individual features let's say So based on the user feedback where the Improvement is expected each feature each feature teams will actually try to improve uh their work over the time and as you go slowly towards you know each quarter you kind of improve your process so things move uh kind of slow but uh uh that is how it should be for the big tech for the large scale unlike startups fair enough yeah I mean if you're

fair enough yeah I mean if you're shipping directly to millions of people worldwide you would definitely want to make sure you have your house in order and you're not just shipping some half baked or not well properly tested uh you know feature that then otherwise breaks people maybe their computers or their brains and yeah that's just obviously PR disaster so I can appreciate that I suppose exactly yeah yeah that that's what I think yeah all the big Tech focuses on meta and everyone Amazon every company true true and um what do you think is the future of let's just start with Adobe Acrobat it's just something that again I've used my entire life doesn't seem to change much solid 9 on 10 gets the job done and you exit and you're good right yeah what is the future here that you EnV Vision it

future here that you EnV Vision it doesn't have to be necessarily you know leaking secrets that you're aware of from Adobe but I'm just trying to get your personal take on where we're headed uh it's more about adding gen again like assistant is one thing Search Assistant these are some of the important components having good search on your PDFs and having good chat Bots and question answering reasoning platform is another thing and then having your generative AI images and editing that would be one thing and if you want to go into the the future and based on how you interacted with them what sort you can do some personalization on your own like uh uh think about user personalization uh so you are a doctor now uh and you have a bunch of documents

now uh and you have a bunch of documents related to doctor uh because you are a doctor you know it you you know this AI knows that you are a doctor so it gives answers based on your level of uh depth that um so and think that is one example and then also you know about chat GPD has its own memory right uh you know it right like has it memory so um so key points that you give to chat GPT it we are allowing it to store it it's it in their memory so internal memory so having a long-term memory always helps uh unless you delete that long-term

uh unless you delete that long-term memory so that can be one another thing good use case think about it's not just one document think about like 100 documents and you have your whole database or set of folder or set of direct R in your local system or any Cloud platform which you wanted to search on which you want to question on so let's imagine there's 100 documents which you want to question on and you don't specify where the answer is within 100 do but you get the answer wow that Cas is not for chat GPT yeah and it it will never be right like chat GPT will never probably I mean it shouldn't um have access to people's you know like local directories or their files so that's a really good good point I had actually not considered how that would

actually not considered how that would really be clutch um and another thing that came to mind when you were talking about the doctor example is you know say I'm a student uh preparing for an exam I have the PDF in front of me and maybe I can ask it to you know generate a question paper based off content over there and I can practice maybe math or whatever I'm studying and you know it can regulate the difficulty level cuz you start slow and then you can be like okay I did nine out of 10 make it harder I think that would also be such a cool you know use case for this similar use case you can generate draft um High

case you can generate draft um High School question paper itself like whole question paper you know uh or undergrad question papers uh the professors can do that like given a of materials this is the level of difficulty or three out of five uh design a paper uh question paper with 10 questions uh where three of them are these three of them are like true or false and then three of them are essay type of questions and it does it for you yeah and even for grading they could probably also use it I mean maybe they shouldn't but if if it were to come to that they could just upload somebody's term paper and we would probably have handwriting to text conversion at a really good spot if it is even

really good spot if it is even handwritten and obviously if it's digital then we're good anyway and then it would just you know grade it for the you know teachers as well so yeah it really does sound like endless um Endless Possibilities here there are actually um applications that use chat gbt to grade actually grade the uh answers um I've seen some platforms do that yeah that's amazing like they they just like let's say there was there are there's one essay written by someone and there's another essay written by someone else so based on the depth or let's say they score it like four out of five or 3.5 out of five it was like amazing yeah no you don't need to read everything you

no you don't need to read everything you just pass it to GP and then you get the answer do have you noticed or have you ever thought about some worrying like obviously this stuff is all so cool and flashy like there's no going around that but does the part of you ever wonder that maybe we've reached a point where or maybe in the future we might reach a point where some of this gets problematic in various ways in that you know students stop to think and apply themselves teachers stop to do what they're supposed to do which is great you know uh papers and all that I guess are you is any part of you a Doomsday when it comes to generative that's a good question uh um it's a mix mix of

good question uh um it's a mix mix of answer so I would say like um using able to use these tools is also a skill so fair yeah yeah but if you want to know how to use them properly you have to have the knowledge already or either should learn from it that way you know what to ask exactly so if you ask right questions you get the right answers gar garbage in is garbage out basically so you have to ask right questions to get right answers so in order to ask right questions you need to have some sort of knowledge um you can't bring some uh uh some person from other domain and ask him to uh uh ask questions on you know uh film making or something like that because they don't know much about it and it will be really

know much about it and it will be really generic like you know their insights and it's it will be very surface level that makes sense yeah guess and prompt engineering and these are other things everyone should learn eventually next two year so that they can ask better questions better ways of asking questions uh to get better answers um yeah but Dooms uh day is is something um because it doesn't kill your creativity it actually improves the way you consume your knowledge uh and it it helps you consume knowledge quickly so that in in one one hour if you're learning x amount of knowledge now you can actually use that to learn 3x 4X if you see it in a right way if you see it in another way it it is bad

you see it in another way it it is bad so there's a mix of an uh mix of uh no that's I I get what you're saying the the thing that trips me personally again this is just very subjectiv um and I've actually done this experiment where you know like just find any book chapter whatever five pages spend time reading those five pages that's you know a and then B is just C copying that entire word all of those words and pasting it into any llm ask it to summarize I find personally that the type of insights that my brain generates for me when I'm reading something you know word to word the oldfashioned way it's funny we have to say that now is very different from what I get from reading that summary like sure it does a good job obviously

like sure it does a good job obviously it is a summary but I find that for me to really absorb information in a deep level it I find it maybe it's just Placebo I don't know but it does lack somehow so yeah have you had any such experiences yeah let's discuss the same example like how I would do because this seems like interesting example um so you you might have just asked okay summarize this book uh based on each chapter yeah or just that chapter like just summarize this chap single chapter yeah so what I would ask is um include all the points where there is like interesting stories think from Human perspective and uh list out each and every point that you feel like this is impactful and then connect it to the uh examples provided in the each the chapter and if there is any sort of

chapter and if there is any sort of story that I have to know or any sort of example that I wanted to know uh you have to Output these examples or you have to also give these examples don't miss out on the story line give the story line in a sequence this is something I would give as a prompt so detail set of instructions then what it does is it will give each and every point that is so important in an order so when you read you feel like okay you have all the context and let's say you had an example of okay uh this person did this so and so did this this person did this so and so and so in the book if you try to summarize it doesn't give that example out right exactly so now if you ask your

out right exactly so now if you ask your real use cases give me use cases uh and don't skip any topic per page don't skip any you just have to prompt it right uh and that way you can get now that llms have so much of a context to fit in they might actually give around half of what is important from that chapter yeah probably more honestly the way you describe that prompt I can see it doing a really good job because I use it fairly often so I I can now by now I'm I'm at a place where I can correlate the type of responses a potential prompt would get I think your prompt would kill

would get I think your prompt would kill like I think it would do a great job yeah yeah that that helps so promp tuning again is it takes prompt generating again takes you to different uh stage actually that's why they pay like 300K for prompt Engineers out there oh I had no idea wow that's that seems wrong is that right I don't know if they're paying now but there was a time in the last year where they said that they were paying like 300 for prompt engineers and everyone was so surprised now now that there are websites and there are tools that generate prompts for you if you give uh prompt and ask how do you want it now there are tools that expands your prompt rites your prom uh rephrases it

prompt rites your prom uh rephrases it like that yeah probably there's a few courses out there as well I'm sure that people could take to get really good at prompt engineering as well like they have to be at this point yeah no matter where they're coming from what time and if they are using Char this is a mustering yeah I really like that actually I feel like that what you just shared really has reshaped pretty much every time I'm going to use chat GPT I'm going to keep that in mind that I need to help it help me like just doing a you know half bake job like you know just do XYZ is not good enough if I go that

XYZ is not good enough if I go that extra mile write those two more sentences that will massively impact what I get out of the m in return so I think that's that's something that has stuck with me I feel like and it will I will be taking that you know going forward and I'm sure our listeners will as well so really appreciate that such a nice Insight that you just shared amazing yeah thanks yeah of course so those are really all the questions I had before we close out here would you is there anything you would like to share with with our listeners um I mean uh ml is huge if you're are rushing uh trying to get a job in J uh just focus on the basics of both ML and J so that and

basics of both ML and J so that and build projects and because it's so underrated people don't build people just do courses I would suggest if you really want to get into gen uh side of things build projects learn learn from the applications and it takes you a long way over the years absolutely n thank you so much for taking the time today it's been such a blast to talk with you not just about your journey but really just um you know brainstorming I think it's the probably the right word uh about generative AI applications and maybe you know some of my ideas were really bad but it was still really fun to bounce them off a you and you know really just pick your brain about this stuff because it does obviously I'm sure

stuff because it does obviously I'm sure that you know this stuff so well and it really just shows so thank you so so much yeah it's really fun talking to you Naman So yeah thank you so much that brings us to the end of episode 35 of the ready said do podcast thank you all for sharing these episodes with those that continue to benefit from them if you would like to support me the easiest way to do that is by subscribing to my YouTube channel leaving me up to a f star rating on Spotify or any other podcast app of your choice or checking out the links in description for additional ways to support me catch you all in the next one new episodes every Wednesday