AI with Claude on AWS: From code to orchestration

Good afternoon everybody. Let's continue with the agenda. Thank you for joining this session. My name is Antonio Rodriguez. I work in Amazon Web Services and I'm also a developer as many of you and also a cloud whisperer as I like to say. We are really just whispering models these days, not really writing code. And uh in this session we are going to be talking about how to take the applications that you are building in CL code from prototypes that you might have in your computer to fully production ready applications that you can have in the cloud in this case with Amazon web services. And uh I think the story that uh we are telling today is a story about teamwork, a story about better together because uh I remember I joined the team and started working uh with anthropic more than three years ago. I remember we were still launching clot 2 as the big amazing model that was uh you know changing and shifting the ground uh below us and u on since then we have tried to combine the best frontier models that anthropic has been developing with the best cloud provider uh that we had in Amazon web services and uh this session by the way is a hands-on session so what we are going to do is that I'm going to give you a brief introduction into how do we work together anthropic and AWS and how you can use clot in AWS in many different ways that we have available today and then we are going to give you some accounts in AWS so you can free of charge and with no limitations test clot in in in bedrock in this case and also in in general in AWS so you can see uh how to use it you have a guided instructions and a workshop and again we are going to give you an environment so you can play around with it. So um again this collaboration started and uh has been continued as a very long-term relationship and um we have uh proven this year by year. Amazon has invested uh has done a multi-billion investment in Antropic uh because we really believe that we are building the future of AI together and at the same time we are the primary cloud provider for Antropic and uh Antropic has uh committed uh more than 100 billion on usage in AWS as well in the infrastructure that we are offering for powering the motors that you are using every day with cloud and u an example of the This investment is project reineer. Project Rineer is actually uh one of the largest uh AI compute infrastructure that was built for training and hosting the cloud models that you are using today with entropy. We are also using uh custom and purpose built chipsets that are specifically made for uh the cloud models in the chipsets that we call Trinium. So we have a our third generation of Trinium chipsets already built uh by Amazon. so that you can actually get the best speed and also use it in the most cost effective way uh when you are expending uh tokens with the with cloud and uh I think that if we start diving into the technical part of uh this collaboration there are three angles that we basically offer you when you use cloud through AWS I think the first one is that you have a very comprehensive list of features that you can use uh for let's say a company the model that you are using. So in bedrock you have a full platform uh so that you can combine and fine-tune the models as well. Uh we are the only provider that allows you to fine-tune haiku in example in the cloud and uh we have full integrations with the rest of the features that we have available in barog and the AWS ecosystem in general. The second and very important point is the security. So we can make sure that you keep control of your data at all times. We avoid this hopping of data going to the public internet. You can keep everything private in your AWS boundary. So we make sure that you uh always align with the compliance and security uh best practices that you might have in your companies or or the regulations. If you are working in banking or healthcare u these kind of industries is uh highly regulated industries. It's very important uh that you actually have these options uh for controlling everything that you're doing with cloud and uh obviously the data privacy part uh we work uh nowadays in the latest generation inference engine that we have in barrack uh we have zero operator access uh in the platform which means that no one from Amazon or from entropic can actually get access to those instances. So your data remains fully private. We are not sharing that data with anyone. No one is using that data for training models or anything like that. So you keep a full restriction and again this is because we need uh to align with a lot of compliance regulations that uh our customers need. There are hundreds of thousands of customers who are using these models every day and therefore it's very important for us to align on those. Last but not least, the scalability. The using those models, cloud models through bedrock gives you u practically infinite scalability in the sense that you can uh really decide how do you want to do the deployments. You have full flexibility for choosing regions as well. In example, you could decide that you want to deploy cloud only in London region in AWS or only in the European region. in example if you have to align with GDPR and and these kind of regulations. Now if we see the whole ecosystem who who is familiar with Amazon bedrock by the way in this room okay so I was suspecting less hands to be honest because uh this is really a clot event but uh it's great to hear that you are using bedrock or you know about bedrock and this and bedrock has been growing a lot since uh the the beginning since the last uh three or four years when we released the service and today apart from the core which is obviously the foundation models that you have access to and obviously uh anthropic cloud models are a very important point part of that. You also have other features that help you in the full journey as a developer. So you have uh features that help you doing evaluation that help you doing pro optimization uh that help you fine-tuning the models as we mentioned before or uh doing model distillation as well if you need it. We also give you some features that help you connecting if you have rack use cases. You can have knowledge bases that are fully managed so that you can attach to it as well. We have uh guard rails which is a way in which you can apply content filters, you can deny topics uh you can uh protect PII data or sensitive data by doing automatic masking and so on. And uh you can also control hallucinations from the models by doing uh proper grounding automated reasoning checks. And uh even more important in the last uh two years I would say tools for building with agentic AI and uh this uh if you are using cloud agents then awesome but if you are using the cloud agent SDK and you are looking for an infrastructure for hosting your agents in a secure way in the cloud then you can also use Amazon bedrock agent core which is uh again fully compatible with any open framework that you might be using for aentic today like chain like crew AI and obviously cloud agency SDK as well. Now if I have to summarize in a single slide everything that I'm saying so far on the why the better together story why is it better to uh use cloud in AWS I would say this is probably the slide right so first of all as I mentioned the data that you can get when you are using cloud in AWS you have a full control of the environment that you are setting up and where you are using it you also have centralized billing from AWS so you can actually have a single build. If your companies are already using AWS or you are using AWS for other things, you can have a consolidated a single build that is actually having the usage of cloud included with everything else that you might be doing today. You have full observability through our services like cloudatch or cloud trail as well in which you can actually see details on metrics logs you can see the traces of everything that has happened and uh we also offer SLAs so we make sure that you have a guarantee availability on the services so you have an uptime that is uh secure for your applications and you can go uh safe to production and make sure that you can scale at the at the cloud scale uh safely ly um another very important point is that we have built-in O integration. So if you are using single sign on or if you are doing open authorization o or if you are using AM roles in AWS in example um you can directly integrate that with your identity provider. You could be using Microsoft enter ID, you could be using octa, you could be using any of those uh identity providers today. And that is a built-in integration that you can do also for using cloud in bedro. And uh last point, you can use private link which means that all the communication can remain fully private. So you make sure that no data is being sent to the public internet at any point. uh when you are using cloud in bad reg as I mentioned I mentioned about compliance before these are some of the compliance regulations that we u support and we are uh good for you to use as well like in example fetram HIPPA and so on and so forth there is a full list online if you want to check it out and uh and more information about all of these topics now the good news is that today you have three ways of using cloud in AWS And actually this is a the the most options that you are going to find in the whole industry right now with first one is you can use cloth models directly through Amazon bedrock as I mentioned before. So basically everything that I was saying so far in which you have the same uh high coetus experience that you might be getting uh directly but in this case uh fully through the Amazon bedrock uh APIs that are offered to you. We have recently added oppos 4.7 and included the messages API as well so that you can directly invoke with the messages API if that's a method that you prefer. Now we have also recently launched cloud platform in on AWS. Cloud platform on AWS is actually generally available since a few days ago and this allows you to have the same exact experience that you might be using today directly with Antropic but in this case with a consolidated billing through AWS and having the access control fully done on AWS as well. We act as a gateway in this case and we send all the inference requests to entropic so that they can process the request for you. So it's giving you the best of both worlds. So you get again the security, the billing, the consolidated access control that we are giving in AWS and at the same time you have all the feature parity with Antropic including everything that we have been hearing in this event and the latest features that you might be using today including web search files agents and so on and so forth. Now the third uh way in which you can use it is also if you are not using programmatically cloud but you are rather using cloud desktop. So you might be using co-work, you might be using uh cloud code through the cloud desktop application. You can also use it directly from AWS. And again you can pay in a consolidated bill in AWS. Uh and that you have two ways of uh doing that as well. One is through the cloud enterprise in in the AWS marketplace in which you pretty much subscribe to the number of developers that you have using this application as well. But you now also have the option of using it uh and and let's say point directly the coord 3P which is one of the latest additions also that the anthropic team did for uh connecting cloud desktop to third parties. In this case you can connect cloud desktop to bedrock or you can connect even cloud desktop to the cloud platform on AWS if you prefer to do so. So again, you have all sort of combinations to make your life easier and uh pretty much meet you where you are on your needs on this. Perfect. Let's dive into the actual workshop that we are going to be building together. Um I just want to uh make a recap that first of all you need two prerequisites for running this workshop. One is you need an AWS account. So that's we got you covered. We are going to give you again a workshop account that has uh an environment that you can play around with and uh you can pretty much be uh free of charge free of any concern of using that environment and then the other thing is that you obviously need to have cloud code installed uh as an application and uh we have a few options for you. So if uh the fox on the back help me switching to my laptop I can show you what is exactly what we are doing. Um you are going to find that in the accounts that we are setting up for you. We are having the workshop that is called introduction to cloud code on AWS and uh this workshop has some instructions to how to set up everything that we are doing. We are pretty much uh playing around with the scaly draw uh tool with the repository of the scaly draw. So you can see how cloud code can help you playing around with a scaly draw and drawing architecture diagrams for you. An example um this is a basic workshop is if you are already an advanced user who who is a getting started with cloud code in this room all right who would say that is a ninja in cloud code so you are you have all the skills on cloud code who is in the middle all right who's not answering so u yeah I guess most of the room is probably on the middle so you will find that probably module one is basic for you So it's a just using cloud code on AWS and learning how to set it up and uh point to scalro and so on. Uh module two and module three are probably more interesting for you. We are going to start uh playing with the context. We are going to start using playright MCP so we can actually uh take a screenshots of the diagram that we are drawing and uh making uh modific automated modifications through cloud code on it. We are also playing around with the uh git workflows and on cloud code and so on. And in the module three we are going to play with sub aents and plugins and we are also creating custom skills hooks and some advanced uh parameters for configuration. I just want to mention that we have an even more advanced workshop that is also available for you if you look for this URL and u basically there you have other things that are very interesting for teams who wants to move to production in a safe way. So here you can learn how to set up u team standards how to implement and and distribute uh advanced workflows as well how to do a scalability cost control and so on. So in example you could see um how to set up dashboards like this one for making sure that you can control all the use that you are doing of uh cloud code tokens uh in your accounts and so on and so forth. Developer productivity measurements uh measuring return of investment of your applications and so on. And last you have also options for using uh the cloud agent SDK on the Amazon BRO agent course. So you can learn how to build the agents connected to the cloud infrastructure uh that we have uh here in in AWS as well. We're coming back to our workshop again. Uh you have different ways of uh doing it. You will see that at the very beginning you will uh see a screen that looks like this with some terms and conditions. I just want to remind everybody that those are workshop accounts. It's uh individual for you but make sure that you don't upload confidential information, personal information or anything like that. You will have to agree to the terms and conditions and click on the join event. And from that point on uh you are going to see pretty much this experience that you have in the instructions here where you actually going to have a an environment fully deployed for you. And uh you will see a URL here that will take you to a terminal and a pretty much a visual studio code UI where you can actually work on all the instructions that we are doing in the workshop. Let's say that you don't want to follow this path and you want to do it in your own machine. Then you can do it. You can use cloud directly in a terminal. If you prefer, you can configure it for pointing to bedrock uh with instructions that we are giving you in the workshop. Let's say that you want to use co-work uh or or the cloud desktop app. Uh you can do it as well. Again, here I'm pointing to my bedrock account. So I have a cloud desktop pointing to bedrock and uh here I'm doing the scaly draw uh application and the and the workshop that we have for you uh over there as well. And uh there is more information at the end. We are going to be sharing with you uh more on a guidance in example for how to use cloud code and co-work on Amazon BRO. There is a full blog from my colleagues uh who is covering all of this in detail and so on and so forth. All right, let's go back to the slides. Uh, please on the back. Thank you. Perfect. So, um, the instructions for accessing the accounts are very easy. All you have to do is go to join.workshop.awws and you have to input that access code that you see on the screen. I think my colleagues uh have been around over there. You have the two uh solutions architects from AWS who can help you with questions. If you have any questions, just raise your hand and we can get uh close and we can help you out. We also have been handing over some uh pieces of paper that has this information so that you can get it uh easier for copy and pasting in in your uh laptops. Um one last comment uh with regards to advanced configurations. If you want to use uh things like in example uh uh configuring the settings that you are using in cloud code manually um this is the kind of parameters that you will need to set up. You might take a picture now if you want. So the here is how to set up uh the region point it to bedrock choose the default model and so on and so forth. And then on the right hand side you also have uh some parameters for reducing the token usage or doing rate limiting and uh how to enable auto reporting for telemetry and uh things like that. You also have some information here about uh uh how to configure extensions in the case that you're using visual studio. Uh but again it is all about asking clot to configure itself or you can do it manually if you prefer through the extensions that we have there. It helps you with some things like an example disable the login prompt that is uh a bit annoying every time that you start or things like uh uh hiding the onboarding uh message that you always get in cloud code and so on and so forth.