crawlee Documentation

Crawlee—A web scraping and browser automation library for Node.js to build reliable crawlers. In JavaScript and TypeScript.

25,363 stars TypeScript TypeScript 1 file · ~14,918 tokens #apify#automation#crawler#crawling#headless#headless-chrome#javascript#nodejs
RAW Doc

1. Project Overview & Quickstart (apify/crawlee)

text
</a>

<small>A web scraping and browser automation library</small>

Crawlee covers your crawling and scraping end-to-end and helps you build reliable scrapers. Fast.

Your crawlers will appear human-like and fly under the radar of modern bot protections even with the default configuration. Crawlee gives you the tools to crawl the web for links, scrape data, and store it to disk or cloud while staying configurable to suit your project's needs.

Crawlee is available as the crawlee NPM package.

👉 View full documentation, guides and examples on the Crawlee project website 👈

Do you prefer 🐍 Python instead of JavaScript? 👉 Checkout Crawlee for Python 👈.

Installation

We recommend visiting the Introduction tutorial in Crawlee documentation for more information.

Crawlee requires Node.js 16 or higher.

With Crawlee CLI

The fastest way to try Crawlee out is to use the Crawlee CLI and choose the Getting started example. The CLI will install all the necessary dependencies and add boilerplate code for you to play with.

bash
npx crawlee create my-crawler
bash
cd my-crawler
npm start

Manual installation

If you prefer adding Crawlee into your own project, try the example below. Because it uses PlaywrightCrawler we also need to install Playwright. It's not bundled with Crawlee to reduce install size.

bash
npm install crawlee playwright
js
import { PlaywrightCrawler, Dataset } from 'crawlee';

// PlaywrightCrawler crawls the web using a headless
// browser controlled by the Playwright library.
const crawler = new PlaywrightCrawler({
    // Use the requestHandler to process each of the crawled pages.
    async requestHandler({ request, page, enqueueLinks, log }) {
        const title = await page.title();
        log.info(`Title of ${request.loadedUrl} is '${title}'`);

        // Save results as JSON to ./storage/datasets/default
        await Dataset.pushData({ title, url: request.loadedUrl });

        // Extract links from the current page
        // and add them to the crawling queue.
        await enqueueLinks();
    },
    // Uncomment this option to see the browser window.
    // headless: false,
});

// Add first URL to the queue and start the crawl.
await crawler.run(['https://crawlee.dev']);

By default, Crawlee stores data to ./storage in the current working directory. You can override this directory via Crawlee configuration. For details, see Configuration guide, Request storage and Result storage.

Installing pre-release versions

We provide automated beta builds for every merged code change in Crawlee. You can find them in the npm list of releases. If you want to test new features or bug fixes before we release them, feel free to install a beta build like this:

bash
npm install crawlee@next

If you also use the Apify SDK, you need to specify dependency overrides in your package.json file so that you don't end up with multiple versions of Crawlee installed:

json
{
    "overrides": {
       "apify": {
           "@crawlee/core": "$crawlee",
           "@crawlee/types": "$crawlee",
           "@crawlee/utils": "$crawlee"
       }
    }
}

🛠 Features

  • Single interface for HTTP and headless browser crawling
  • Persistent queue for URLs to crawl (breadth & depth first)
  • Pluggable storage of both tabular data and files
  • Automatic scaling with available system resources
  • Integrated proxy rotation and session management
  • Lifecycles customizable with hooks
  • CLI to bootstrap your projects
  • Configurable routing, error handling and retries
  • Dockerfiles ready to deploy
  • Written in TypeScript with generics

👾 HTTP crawling

  • Zero config HTTP2 support, even for proxies
  • Automatic generation of browser-like headers
  • Replication of browser TLS fingerprints
  • Integrated fast HTML parsers. Cheerio and JSDOM
  • Yes, you can scrape JSON APIs as well

💻 Real browser crawling

  • JavaScript rendering and screenshots
  • Headless and headful support
  • Zero-config generation of human-like fingerprints
  • Automatic browser management
  • Use Playwright and Puppeteer with the same interface
  • Chrome, Firefox, Webkit and many others

Usage on the Apify platform

Crawlee is open-source and runs anywhere, but since it's developed by Apify, it's easy to set up on the Apify platform and run in the cloud. Visit the Apify SDK website to learn more about deploying Crawlee to the Apify platform.

Support

If you find any bug or issue with Crawlee, please submit an issue on GitHub. For questions, you can ask on Stack Overflow, in GitHub Discussions or you can join our Discord server.

Contributing

Your code contributions are welcome, and you'll be praised to eternity! If you have any ideas for improvements, either submit an issue or create a pull request. For contribution guidelines and the code of conduct, see CONTRIBUTING.md.

License

This project is licensed under the Apache License 2.0 - see the LICENSE.md file for details.

2. Official Technical Reference & Guides (apify/docs)

File: README.md

Strands Agents Documentation

A model-driven approach to building AI agents in just a few lines of code.

text
[Documentation](https://strandsagents.com/)
◆ [Samples](https://github.com/strands-agents/samples)
◆ [Python SDK](https://github.com/strands-agents/sdk-python)
◆ [Tools](https://github.com/strands-agents/tools)
◆ [Agent Builder](https://github.com/strands-agents/agent-builder)
◆ [MCP Server](https://github.com/strands-agents/mcp-server)

This repository contains the documentation for the Strands Agents SDK, a simple yet powerful framework for building and running AI agents. The documentation is built using Astro with the Starlight theme and provides guides, examples, and API references.

The official documentation is available online at: https://strandsagents.com.

Local Development

Prerequisites

  • Python 3.10+
  • Node.js 20+, npm

Setup and Installation

bash
npm install

Building and Previewing

Generate the static site:

bash
npm run build

Run a local development server at http://localhost:4321/:

bash
npm run dev

Writing Documentation

Documentation lives in docs/ as Markdown files. The site structure is driven by src/config/navigation.yml (navigation) and rendered by Astro at build time.

  • Pages are written in standard Markdown — no Astro-specific syntax needed for content edits
  • Use <Tabs> / <Tab label="..."> for language-switching code blocks (auto-imported, no import needed)
  • Use --8<-- "path/to/file.ts:snippet_name" to include code snippets from external files
  • Link to other pages using relative file paths (e.g. ../tools/index.md) — they resolve automatically
  • Link to API reference pages using the @api shorthand: [@api/python/strands.agent.agent](#AgentResult)

For a full reference on customizations, components, and the migration status, see Site Architecture.

Contributing ❤️

We welcome contributions! See our Contributing Guide for details on:

  • Reporting bugs & features
  • Development setup
  • Contributing via Pull Requests
  • Code of Conduct
  • Reporting of security issues

License

This project is licensed under the Apache License 2.0 - see the LICENSE file for details.

Security

See CONTRIBUTING for more information.

File: designs/README.md

Design Documents

This folder contains design documents for significant features and changes to Strands Agents. These documents capture the context, decision, and consequences of architectural choices.

For lightweight architecture decision records, see DECISIONS.md.

What is a design document?

A design document proposes a significant change or new feature. It describes the problem, proposed solution, and tradeoffs. Once approved and merged, it becomes an accepted design and part of the project's decision history.

When to write a design document

Write a design document for:

  • New major features affecting multiple parts of the SDK
  • Breaking changes to existing APIs
  • Architectural changes requiring design discussion
  • Large contributions (> 1 week of work)
  • Features that introduce new concepts

Skip the design process for bug fixes, small improvements, documentation updates, and new extensions in your own repository.

How to submit a feature proposal

  1. Check the roadmap — See if your idea aligns with our direction
  2. Fork the docs repository
  3. Create your design — Add a new file: designs/NNNN-feature-name.md using the template below
  4. Submit a pull request — We'll review and discuss
  5. Iterate based on feedback — Address comments and questions
  6. Get approval — Once approved and merged, implement the feature
  7. Reference the design — Link to it in your implementation PR

Design document template

markdown
# [Feature Name]

**Status**: Proposed | Accepted | Deprecated | Superseded

**Date**: YYYY-MM-DD

**Issue**: Issue Link

## Context

What is the issue that we're seeing that is motivating this decision or change?

- What task are you trying to accomplish?
- What makes it difficult or impossible today?
- Who experiences this problem?

## Decision

What is the change that we're proposing and/or doing?

- Changes to the API (with code examples)
- How it integrates with existing features
- Expected behavior and usage patterns

## Developer Experience

Show what the developer experience looks like:

- Code examples showing typical usage
- Configuration or setup required
- Error messages and edge cases

## Alternatives Considered

What other approaches did you consider? Why did you choose this one?

## Consequences

What becomes easier or more difficult to do because of this change?

## Willingness to Implement

Are you willing to implement this if approved?

Yes / No / Maybe with guidance

Accepted Designs

No designs have been accepted yet.

File: docs/examples/typescript/deploy_to_bedrock_agentcore/README.md

TypeScript Agent Deployment to Amazon Bedrock AgentCore Runtime

This example demonstrates deploying a TypeScript-based Strands agent to Amazon Bedrock AgentCore Runtime using Express and Docker.

What's Included

This example includes a complete, ready-to-deploy agent service with:

  • Express-based HTTP server with required AgentCore endpoints (/ping and /invocations)
  • Calculator tool demonstrating custom tool implementation
  • Amazon Bedrock integration for LLM inference
  • Docker configuration for containerized deployment via AgentCore
  • IAM role automation scripts for AWS permissions setup
  • Test script for invoking the deployed agent

Prerequisites

Before you begin, ensure you have:

  • Node.js 20+
  • Docker installed and running
  • AWS CLI configured with valid credentials
  • AWS account with appropriate permissions
  • ECR repository access

Project Structure

text
.
├── index.ts               # Main agent service implementation
├── invoke.ts              # Test script for deployed agent
├── package.json           # Node.js dependencies and scripts
├── tsconfig.json          # TypeScript configuration
├── Dockerfile             # Container configuration
├── create-iam-role.sh     # IAM role automation script
└── README.md              # This file

Quick Start

1. Install Dependencies

bash
npm install

2. Test Locally

Build and start the server:

bash
npm run build
npm start

In another terminal, test the health check:

bash
curl http://localhost:8080/ping

Test the agent:

bash
echo -n "What is 5 plus 3?" | curl -X POST http://localhost:8080/invocations \
  -H "Content-Type: application/octet-stream" \
  --data-binary @-

3. Test with Docker

Build the Docker image:

bash
docker build -t my-agent-service .

Run the container:

bash
docker run -p 8081:8080 my-agent-service

Test in another terminal:

bash
curl http://localhost:8081/ping

Deployment to AWS

Step 1: Create IAM Role

Option A: Automated Script (Recommended)

Make the script executable and run it:

bash
chmod +x create-iam-role.sh
./create-iam-role.sh

The script will output the Role ARN. Save this for deployment.

Option B: Manual Setup

Create the role manually using AWS CLI or Console following the steps outlined in the above script.

Step 2: Set Environment Variables

bash
# Get your AWS Account ID
export ACCOUNTID=$(aws sts get-caller-identity --query Account --output text)

# Set your preferred region
export AWS_REGION=ap-southeast-2

# Get the IAM Role ARN
export ROLE_ARN=$(aws iam get-role \
  --role-name BedrockAgentCoreRuntimeRole \
  --query 'Role.Arn' \
  --output text)

# Set ECR repository name
export ECR_REPO=my-agent-service

Step 3: Create ECR Repository

bash
aws ecr create-repository \
  --repository-name ${ECR_REPO} \
  --region ${AWS_REGION}

Step 4: Build and Push Docker Image

Login to ECR:

bash
aws ecr get-login-password --region ${AWS_REGION} | \
  docker login --username AWS --password-stdin \
  ${ACCOUNTID}.dkr.ecr.${AWS_REGION}.amazonaws.com

Build, Tag, and push:

bash
docker build -t ${ECR_REPO} .

docker tag ${ECR_REPO}:latest \
  ${ACCOUNTID}.dkr.ecr.${AWS_REGION}.amazonaws.com/${ECR_REPO}:latest

docker push ${ACCOUNTID}.dkr.ecr.${AWS_REGION}.amazonaws.com/${ECR_REPO}:latest

Step 5: Create AgentCore Runtime

bash
aws bedrock-agentcore-control create-agent-runtime \
  --agent-runtime-name my_agent_service \
  --agent-runtime-artifact containerConfiguration={containerUri=${ACCOUNTID}.dkr.ecr.${AWS_REGION}.amazonaws.com/${ECR_REPO}:latest} \
  --role-arn ${ROLE_ARN} \
  --network-configuration networkMode=PUBLIC \
  --protocol-configuration serverProtocol=HTTP \
  --region ${AWS_REGION}

Step 6: Verify Deployment

Wait about a minute, then check the status:

bash
aws bedrock-agentcore-control get-agent-runtime \
  --agent-runtime-id my-agent-service-XXXXXXXXXX \
  --region ${AWS_REGION} \
  --query 'status' \
  --output text

Replace XXXXXXXXXX with your runtime ID from the create command output.

Step 7: Test Your Deployment

  1. Update invoke.ts with your AWS Account ID and runtime ID
  2. Run the test:
bash
npm run test:invoke

Expected output:

text
Response: {"response":{"type":"agentResult","stopReason":"endTurn",...}}

Customization

Adding More Tools

Add custom tools to the agent configuration in index.ts:

typescript
const myCustomTool = strands.tool({
  name: 'my_tool',
  inputSchema: z.object({
    // Define your input schema
  }),
  callback: (input) => {
    // Implement your tool logic
  },
})

const agent = new strands.Agent({
  model: new strands.BedrockModel({
    region: 'ap-southeast-2',
  }),
  tools: [calculatorTool, myCustomTool], // Add your tool here
})

Updating Your Deployment

After making code changes:

  1. Build and push new Docker image:
bash
docker build -t ${ACCOUNTID}.dkr.ecr.${AWS_REGION}.amazonaws.com/${ECR_REPO}:latest . --no-cache
docker push ${ACCOUNTID}.dkr.ecr.${AWS_REGION}.amazonaws.com/${ECR_REPO}:latest
  1. Update the runtime:
bash
aws bedrock-agentcore-control update-agent-runtime \
  --agent-runtime-id "my-agent-service-XXXXXXXXXX" \
  --agent-runtime-artifact "{\"containerConfiguration\": {\"containerUri\": \"${ACCOUNTID}.dkr.ecr.${AWS_REGION}.amazonaws.com/${ECR_REPO}:latest\"}}" \
  --role-arn "${ROLE_ARN}" \
  --network-configuration "{\"networkMode\": \"PUBLIC\"}" \
  --protocol-configuration serverProtocol=HTTP \
  --region ${AWS_REGION}
  1. Wait a minute and test with npm run test:invoke

Troubleshooting

TypeScript Compilation Errors

Clean and rebuild:

bash
rm -rf dist node_modules
npm install
npm run build

Docker Build Fails

Ensure Docker is running:

bash
docker info

Build without cache:

bash
docker build --no-cache -t my-agent-service .

ECR Authentication Expired

Re-authenticate:

bash
aws ecr get-login-password --region ${AWS_REGION} | \
  docker login --username AWS --password-stdin \
  ${ACCOUNTID}.dkr.ecr.${AWS_REGION}.amazonaws.com

View CloudWatch Logs

bash
aws logs tail /aws/bedrock-agentcore/runtimes/my-agent-service-XXXXXXXXXX-DEFAULT \
  --region ${AWS_REGION} \
  --since 1h \
  --follow

Additional Resources

Support

For issues or questions:

File: docs/examples/python/multi_agent_example/index.md

Multi-Agent Example

This directory contains the implementation files for the Multi-Agent Example architecture, where specialized agents work together under the coordination of a central orchestrator.

Implementation Files

Documentation

For detailed information about how this multi-agent architecture works, please see the multi_agent_example.md documentation file.

File: docs/examples/deploy_to_eks/README.md

Amazon EKS Deployment Example

Introduction

This is an example that demonstrates how to deploy a Python application to Amazon EKS.
The example deploys a weather forecaster application that runs as a containerized service in Amazon EKS with an Application Load Balancer. The application is built with FastAPI and provides two weather endpoints:

  1. /weather - A standard endpoint that returns weather information based on the provided prompt
  2. /weather-streaming - A streaming endpoint that delivers weather information in real-time as it's being generated

Prerequisites

  • AWS CLI installed and configured
  • eksctl (v0.208.x or later) installed
  • Helm (v3 or later) installed
  • kubectl installed
  • Either:
    • Podman installed and running
    • (or) Docker installed and running
  • Amazon Bedrock Anthropic Claude 4 model enabled in your AWS environment

Project Structure

  • chart/ - Contains the Helm chart
    • values.yaml - Helm chart default values
  • docker/ - Contains the Dockerfile and application code for the container:
    • Dockerfile - Docker image definition
    • app/ - Application code
    • requirements.txt - Python dependencies for the container & local development

Create EKS Auto Mode cluster

Set environment variables

bash
export AWS_ACCOUNT_ID=$(aws sts get-caller-identity --query 'Account' --output text)
export AWS_REGION=us-east-1
export CLUSTER_NAME=eks-strands-agents-demo

Create EKS Auto Mode cluster

bash
eksctl create cluster --name $CLUSTER_NAME --enable-auto-mode

Configure kubeconfig context

bash
aws eks update-kubeconfig --name $CLUSTER_NAME

Building and Pushing Docker Image to ECR

Follow these steps to build the Docker image and push it to Amazon ECR:

  1. Authenticate to Amazon ECR:
bash
aws ecr get-login-password --region ${AWS_REGION} | docker login --username AWS --password-stdin ${AWS_ACCOUNT_ID}.dkr.ecr.${AWS_REGION}.amazonaws.com
  1. Create the ECR repository if it doesn't exist:
bash
aws ecr create-repository --repository-name strands-agents-weather --region ${AWS_REGION}
  1. Build the Docker image:
bash
docker build --platform linux/amd64 -t strands-agents-weather:latest docker/
  1. Tag the image for ECR:
bash
docker tag strands-agents-weather:latest ${AWS_ACCOUNT_ID}.dkr.ecr.${AWS_REGION}.amazonaws.com/strands-agents-weather:latest
  1. Push the image to ECR:
bash
docker push ${AWS_ACCOUNT_ID}.dkr.ecr.${AWS_REGION}.amazonaws.com/strands-agents-weather:latest

Configure EKS Pod Identity to access Amazon Bedrock

Create an IAM policy to allow InvokeModel & InvokeModelWithResponseStream to all Amazon Bedrock models

bash
cat > bedrock-policy.json << EOF
{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": [
        "bedrock:InvokeModel",
        "bedrock:InvokeModelWithResponseStream"
      ],
      "Resource": "*"
    }
  ]
}
EOF

aws iam create-policy \
  --policy-name strands-agents-weather-bedrock-policy \
  --policy-document file://bedrock-policy.json
rm -f bedrock-policy.json

Create an EKS Pod Identity association

bash
eksctl create podidentityassociation --cluster $CLUSTER_NAME \
  --namespace default \
  --service-account-name strands-agents-weather \
  --permission-policy-arns arn:aws:iam::$AWS_ACCOUNT_ID:policy/strands-agents-weather-bedrock-policy \
  --role-name eks-strands-agents-weather

Deploy strands-agents-weather application

Deploy the helm chart with the image from ECR

bash
helm install strands-agents-weather ./chart \
  --set image.repository=${AWS_ACCOUNT_ID}.dkr.ecr.${AWS_REGION}.amazonaws.com/strands-agents-weather --set image.tag=latest

Wait for Deployment to be available (Pods Running)

bash
kubectl wait --for=condition=available deployments strands-agents-weather --all

Test the Agent

Using kubernetes port-forward

text
kubectl --namespace default port-forward service/strands-agents-weather 8080:80 &

Call the weather service

text
curl -X POST \
  http://localhost:8080/weather \
  -H 'Content-Type: application/json' \
  -d '{"prompt": "What is the weather in Seattle?"}'

Call the weather streaming endpoint

text
curl -X POST \
  http://localhost:8080/weather-streaming \
  -H 'Content-Type: application/json' \
  -d '{"prompt": "What is the weather in New York in Celsius?"}'

Expose Agent through Application Load Balancer

Create an IngressClass to configure an Application Load Balancer

bash
cat <<EOF | kubectl apply -f -
apiVersion: eks.amazonaws.com/v1
kind: IngressClassParams
metadata:
  name: alb
spec:
  scheme: internet-facing
EOF
bash
cat <<EOF | kubectl apply -f -
apiVersion: networking.k8s.io/v1
kind: IngressClass
metadata:
  name: alb
  annotations:
    ingressclass.kubernetes.io/is-default-class: "true"
spec:
  controller: eks.amazonaws.com/alb
  parameters:
    apiGroup: eks.amazonaws.com
    kind: IngressClassParams
    name: alb
EOF

Update helm deployment to create Ingress using the IngressClass created

bash
helm upgrade strands-agents-weather ./chart \
  --set image.repository=${AWS_ACCOUNT_ID}.dkr.ecr.${AWS_REGION}.amazonaws.com/strands-agents-weather --set image.tag=latest \
  --set ingress.enabled=true \
  --set ingress.className=alb

Get the ALB URL

bash
export ALB_URL=$(kubectl get ingress strands-agents-weather -o jsonpath='{.status.loadBalancer.ingress[0].hostname}')
echo "The shared ALB is available at: http://$ALB_URL"

Wait for ALB to be active

bash
aws elbv2 wait load-balancer-available --load-balancer-arns $(aws elbv2 describe-load-balancers --query 'LoadBalancers[?DNSName==`'"$ALB_URL"'`].LoadBalancerArn' --output text)

Call the weather service Application Load Balancer endpoint

bash
curl -X POST \
  http://$ALB_URL/weather \
  -H 'Content-Type: application/json' \
  -d '{"prompt": "What is the weather in Portland?"}'

Configure High Availability and Resiliency

bash
helm upgrade strands-agents-weather ./chart -f - <<EOF
image:
  tag: latest

ingress:
  enabled: true 
  className: alb

replicaCount: 3

topologySpreadConstraints:
  - maxSkew: 1
    minDomains: 3
    topologyKey: topology.kubernetes.io/zone
    whenUnsatisfiable: DoNotSchedule
    labelSelector:
      matchLabels:
        app.kubernetes.io/name: strands-agents-weather
  - maxSkew: 1
    topologyKey: kubernetes.io/hostname
    whenUnsatisfiable: ScheduleAnyway
    labelSelector:
      matchLabels:
        app.kubernetes.io/instance: strands-agents-weather
          
podDisruptionBudget:
  enabled: true
  minAvailable: 1
EOF

Cleanup

Uninstall helm chart

bash
helm uninstall strands-agents-weather

Delete EKS Auto Mode cluster

bash
eksctl delete cluster --name $CLUSTER_NAME --wait

Delete IAM policy

bash
aws iam delete-policy --policy-arn arn:aws:iam::$AWS_ACCOUNT_ID:policy/strands-agents-weather-bedrock-policy

File: docs/examples/cdk/deploy_to_lambda/README.md

AWS CDK Lambda Deployment Example

Introduction

This is a TypeScript-based CDK (Cloud Development Kit) example that demonstrates how to deploy a Python function to AWS Lambda. The example deploys a weather forecaster application that requires AWS authentication to invoke the Lambda function.

Prerequisites

  • AWS CLI installed and configured
  • Node.js (v18.x or later)
  • Python 3.12 or later
  • jq (optional) for formatting JSON output

Project Structure

  • lib/ - Contains the CDK stack definition in TypeScript
  • bin/ - Contains the CDK app entry point and deployment scripts:
    • cdk-app.ts - Main CDK application entry point
    • package_for_lambda.py - Python script that packages Lambda code and dependencies into deployment archives
  • lambda/ - Contains the Python Lambda function code
  • packaging/ - Directory used to store Lambda deployment assets and dependencies

Setup and Deployment

  1. Install dependencies:
bash
# Install Node.js dependencies including CDK and TypeScript locally
npm install

# Create a Python virtual environment (optional but recommended)
python -m venv .venv
source .venv/bin/activate  # On Windows: .venv\Scripts\activate

# Install Python dependencies for the local development
pip install -r requirements.txt
# Install Python dependencies for lambda with correct architecture
pip install -r requirements.txt --python-version 3.12 --platform manylinux2014_aarch64 --target ./packaging/_dependencies --only-binary=:all:
  1. Package the lambda:
bash
python ./bin/package_for_lambda.py
  1. Bootstrap your AWS environment (if not already done):
bash
npx cdk bootstrap
  1. Deploy the lambda:
text
npx cdk deploy

Usage

After deployment, you can invoke the Lambda function using the AWS CLI or AWS Console. The function requires proper AWS authentication to be invoked.

bash
aws lambda invoke --function-name AgentFunction \
      --region us-east-1 \
      --cli-binary-format raw-in-base64-out \
      --payload '{"prompt": "What is the weather in New York?"}' \
      output.json

If you have jq installed, you can output the response from output.json like so:

bash
jq -r '.' ./output.json

Otherwise, open output.json to view the result.

Cleanup

To remove all resources created by this example:

bash
npx cdk destroy

Additional Resources

File: docs/examples/cdk/deploy_to_fargate/README.md

AWS CDK Fargate Deployment Example

Introduction

This is a TypeScript-based CDK (Cloud Development Kit) example that demonstrates how to deploy a Python application to AWS Fargate. The example deploys a weather forecaster application that runs as a containerized service in AWS Fargate with an Application Load Balancer. The application is built with FastAPI and provides two weather endpoints:

  1. /weather - A standard endpoint that returns weather information based on the provided prompt
  2. /weather-streaming - A streaming endpoint that delivers weather information in real-time as it's being generated

Prerequisites

  • AWS CLI installed and configured
  • Node.js (v18.x or later)
  • Python 3.12 or later
  • Either:
    • Podman installed and running
    • (or) Docker installed and running

Project Structure

  • lib/ - Contains the CDK stack definition in TypeScript
  • bin/ - Contains the CDK app entry point and deployment scripts:
    • cdk-app.ts - Main CDK application entry point
  • docker/ - Contains the Dockerfile and application code for the container:
    • Dockerfile - Docker image definition
    • app/ - Application code
    • requirements.txt - Python dependencies for the container & local development

Setup and Deployment

  1. Install dependencies:
bash
# Install Node.js dependencies including CDK and TypeScript locally
npm install

# Create a Python virtual environment (optional but recommended)
python -m venv .venv
source .venv/bin/activate  # On Windows: .venv\Scripts\activate

# Install Python dependencies for the local development
pip install -r ./docker/requirements.txt
  1. Bootstrap your AWS environment (if not already done):
bash
npx cdk bootstrap
  1. Ensure podman is started (one time):
bash
podman machine init
podman machine start
  1. Package & deploy via CDK:
bash
CDK_DOCKER=podman npx cdk deploy

Usage

After deployment, you can access the weather service using the Application Load Balancer URL that is output after deployment:

bash
# Get the service URL from the CDK output
SERVICE_URL=$(aws cloudformation describe-stacks --stack-name AgentFargateStack --query "Stacks[0].Outputs[?ExportName=='AgentServiceEndpoint'].OutputValue" --output text)

The service exposes a REST API endpoint that you can call using curl or any HTTP client:

bash
# Call the weather service
curl -X POST \
  http://$SERVICE_URL/weather \
  -H 'Content-Type: application/json' \
  -d '{"prompt": "What is the weather in New York?"}'
  
 # Call the streaming endpoint
 curl -X POST \
  http://$SERVICE_URL/weather-streaming \
  -H 'Content-Type: application/json' \
  -d '{"prompt": "What is the weather in New York in Celsius?"}'

Local testing (python)

You can run the python app directly for local testing via:

bash
python ./docker/app/app.py

Then, set the SERVICE_URL to point to your local server

bash
SERVICE_URL=127.0.0.1:8000

and you can use the curl commands above to test locally.

Local testing (container)

Build & run the container:

bash
podman build ./docker/ -t agent_container
podman run -p 127.0.0.1:8000:8000 -t agent_container

Then, set the SERVICE_URL to point to your local server

bash
SERVICE_URL=127.0.0.1:8000

and you can use the curl commands above to test locally.

Cleanup

To remove all resources created by this example:

bash
npx cdk destroy

Additional Resources

File: docs/examples/cdk/deploy_to_ec2/README.md

AWS CDK EC2 Deployment Example

Introduction

This is a TypeScript-based CDK (Cloud Development Kit) example that demonstrates how to deploy a Python application to AWS EC2. The example deploys a weather forecaster application that runs as a service on an EC2 instance. The application provides two weather endpoints:

  1. /weather - A standard endpoint that returns weather information based on the provided prompt
  2. /weather-streaming - A streaming endpoint that delivers weather information in real-time as it's being generated

Prerequisites

  • AWS CLI installed and configured
  • Node.js (v18.x or later)
  • Python 3.12 or later

Project Structure

  • lib/ - Contains the CDK stack definition in TypeScript
  • bin/ - Contains the CDK app entry point and deployment scripts:
    • cdk-app.ts - Main CDK application entry point
  • app/ - Contains the application code:
    • app.py - FastAPI application code
  • requirements.txt - Python dependencies for the application

Setup and Deployment

  1. Install dependencies:
bash
# Install Node.js dependencies including CDK and TypeScript locally
npm install

# Create a Python virtual environment (optional but recommended)
python -m venv .venv
source .venv/bin/activate  # On Windows: .venv\Scripts\activate

# Install Python dependencies for the local development
pip install -r ./requirements.txt

# Install Python dependencies for the app distribution
pip install -r requirements.txt --python-version 3.12 --platform manylinux2014_aarch64 --target ./packaging/_dependencies --only-binary=:all:
  1. Bootstrap your AWS environment (if not already done):
bash
npx cdk bootstrap
  1. Deploy the stack:
bash
npx cdk deploy

How It Works

This deployment:

  1. Creates an EC2 instance in a public subnet with a public IP
  2. Uploads the application code to S3 as CDK assets
  3. Uses a user data script to:
    • Install Python and other dependencies
    • Download the application code from S3
    • Set up the application as a systemd service using uvicorn

Usage

After deployment, you can access the weather service using the Application Load Balancer URL that is output after deployment:

bash
# Get the service URL from the CDK output
SERVICE_URL=$(aws cloudformation describe-stacks --stack-name AgentEC2Stack --region us-east-1 --query "Stacks[0].Outputs[?ExportName=='Ec2ServiceEndpoint'].OutputValue" --output text)

The service exposes a REST API endpoint that you can call using curl or any HTTP client:

bash
# Call the weather service
curl -X POST \
  http://$SERVICE_URL/weather \
  -H 'Content-Type: application/json' \
  -d '{"prompt": "What is the weather in New York?"}'
  
 # Call the streaming endpoint
 curl -X POST \
  http://$SERVICE_URL/weather-streaming \
  -H 'Content-Type: application/json' \
  -d '{"prompt": "What is the weather in New York in Celsius?"}'

Local testing

You can run the python app directly for local testing via:

bash
python app/app.py

Then, set the SERVICE_URL to point to your local server

bash
SERVICE_URL=127.0.0.1:8000

and you can use the curl commands above to test locally.

Cleanup

To remove all resources created by this example:

bash
npx cdk destroy

Callouts and considerations

Note that this example demonstrates a simple deployment approach with some important limitations:

  • The application code is deployed only during the initial instance creation via user data script
  • Updating the application requires implementing a custom update mechanism
  • The example exposes the application directly on port 8000 without a load balancer
  • For production workloads, consider using ECS/Fargate/App Runner which provides built-in support for application updates, scaling, and high availability

Additional Resources

File: docs/examples/cdk/deploy_to_apprunner/README.md

AWS CDK App Runner Deployment Example

Introduction

This is a TypeScript-based CDK (Cloud Development Kit) example that demonstrates how to deploy a Python application to AWS App Runner. The example deploys a weather forecaster application that runs as a containerized service in AWS App Runner. The application is built with FastAPI and provides two weather endpoints:

  1. /weather - A standard endpoint that returns weather information based on the provided prompt
  2. /weather-streaming - A streaming endpoint that delivers weather information in real-time as it's being generated

Prerequisites

  • AWS CLI installed and configured
  • Node.js (v18.x or later)
  • Python 3.12 or later
  • Either:
    • Podman installed and running
    • (or) Docker installed and running

Project Structure

  • lib/ - Contains the CDK stack definition in TypeScript
  • bin/ - Contains the CDK app entry point and deployment scripts:
    • cdk-app.ts - Main CDK application entry point
  • docker/ - Contains the Dockerfile and application code for the container:
    • Dockerfile - Docker image definition
    • app/ - Application code
    • requirements.txt - Python dependencies for the container & local development

Setup and Deployment

  1. Install dependencies:
bash
# Install Node.js dependencies including CDK and TypeScript locally
npm install

# Create a Python virtual environment (optional but recommended)
python -m venv .venv
source .venv/bin/activate  # On Windows: .venv\Scripts\activate

# Install Python dependencies for the local development
pip install -r ./docker/requirements.txt
  1. Bootstrap your AWS environment (if not already done):
bash
npx cdk bootstrap
  1. Ensure podman is started (one time):
bash
podman machine init
podman machine start
  1. Package & deploy via CDK:
bash
CDK_DOCKER=podman npx cdk deploy

Usage

After deployment, you can access the weather service using the App Runner Service URL that is output after deployment:

bash
# Get the service URL from the CDK output
SERVICE_URL=$(aws cloudformation describe-stacks --stack-name AgentAppRunnerStack --query "Stacks[0].Outputs[?ExportName=='AppRunnerServiceUrl'].OutputValue" --output text)

The service exposes a REST API endpoint that you can call using curl or any HTTP client:

bash
# Call the weather service
curl -X POST \
  https://$SERVICE_URL/weather \
  -H 'Content-Type: application/json' \
  -d '{"prompt": "What is the weather in New York?"}'
  
# Call the streaming endpoint
curl -X POST \
  https://$SERVICE_URL/weather-streaming \
  -H 'Content-Type: application/json' \
  -d '{"prompt": "What is the weather in New York in Celsius?"}'

Local testing (python)

You can run the python app directly for local testing via:

bash
python ./docker/app/app.py

Then, set the SERVICE_URL to point to your local server

bash
SERVICE_URL=127.0.0.1:8000

and you can use the curl commands above to test locally.

Local testing (container)

Build & run the container:

bash
podman build --platform linux/amd64 -t agent_container ./docker/

podman run -p 127.0.0.1:8000:8000 \
  -v ~/.aws:/home/appuser/.aws:ro \
  -e AWS_PROFILE=default \
  -t agent_container

Then, set the SERVICE_URL to point to your local server

bash
SERVICE_URL=127.0.0.1:8000

and you can use the curl commands above to test locally.

Cleanup

To remove all resources created by this example:

bash
npx cdk destroy

Additional Resources

File: src/content/blog/introducing-strands-agents.mdx

title: "Introducing Strands Agents, an Open Source AI Agents SDK"
date: 2025-05-16
authors:

Today I am happy to announce we are releasing Strands Agents. Strands Agents is an open source SDK that takes a model-driven approach to building and running AI agents in just a few lines of code. Strands scales from simple to complex agent use cases, and from local development to deployment in production. Multiple teams at AWS already use Strands for their AI agents in production, including Amazon Q Developer, AWS Glue, and VPC Reachability Analyzer. Now, I'm thrilled to share Strands with you for building your own AI agents.

Compared with frameworks that require developers to define complex workflows for their agents, Strands simplifies agent development by embracing the capabilities of state-of-the-art models to plan, chain thoughts, call tools, and reflect. With Strands, developers can simply define a prompt and a list of tools in code to build an agent, then test it locally and deploy it to the cloud. Like the two strands of DNA, Strands connects two core pieces of the agent together: the model and the tools. Strands plans the agent's next steps and executes tools using the advanced reasoning capabilities of models. For more complex agent use cases, developers can customize their agent's behavior in Strands. For example, you can specify how tools are selected, customize how context is managed, choose where session state and memory are stored, and build multi-agent applications. Strands can run anywhere and can support any model with reasoning and tool use capabilities, including models in Amazon Bedrock, Anthropic, Ollama, Meta, and other providers through LiteLLM.

Strands Agents is an open community, and we're excited that several companies are joining us with support and contributions including Accenture, Anthropic, Langfuse, mem0.ai, Meta, PwC, Ragas.io, and Tavily. For instance, Anthropic has already contributed support in Strands for using models through the Anthropic API, and Meta contributed support for Llama models through Llama API. Join us on GitHub to get started with Strands Agents!

Our journey building agents

I primarily work on Amazon Q Developer, a generative AI-powered assistant for software development. My team and I started building AI agents in early 2023, around when the original ReAct (Reasoning and Acting) scientific paper was published. This paper showed that large language models could reason, plan, and take actions in their environment. For example, LLMs could reason that they needed to make an API call to complete a task and then generate the inputs needed for that API call. We then realized that large language models could be used as agents to complete many types of tasks, including complex software development and operational troubleshooting.

At that time, LLMs weren't typically trained to act like agents. They were often trained primarily for natural language conversation. Successfully using an LLM to reason and act required complex prompt instructions on how to use tools, parsers for the model's responses, and orchestration logic. Simply getting LLMs to reliably produce syntactically correct JSON was a challenge at the time! To prototype and deploy agents, my team and I relied on a variety of complex agent framework libraries that handled the scaffolding and orchestration needed for the agents to reliably succeed at their tasks with these earlier models. Even with these frameworks, it would take us months of tuning and tweaking to get an agent ready for production.

Since then, we've seen a dramatic improvement in large language models' abilities to reason and use tools to complete tasks. We realized that we no longer needed such complex orchestration to build agents, because models now have native tool-use and reasoning capabilities. In fact, some of the agent framework libraries we had been using to build our agents started to get in our way of fully leveraging the capabilities of newer LLMs. Even though LLMs were getting dramatically better, those improvements didn't mean we could build and iterate on agents any faster with the frameworks we were using. It still took us months to make an agent production-ready.

We started building Strands Agents to remove this complexity for our teams in Q Developer. We found that relying on the latest models' capabilities to drive agents significantly reduced our time to market and improved the end user experience, compared to building agents with complex orchestration logic. Where it used to take months for Q Developer teams to go from prototype to production with a new agent, we're now able to ship new agents in days and weeks with Strands.

Core concepts of Strands Agents

The simplest definition of an agent is a combination of three things: 1) a model, 2) tools, and 3) a prompt. The agent uses these three components to complete a task, often autonomously. The agent's task could be to answer a question, generate code, plan a vacation, or optimize your financial portfolio. In a model-driven approach, the agent uses the model to dynamically direct its own steps and to use tools in order to accomplish the specified task.

To define an agent with the Strands Agents SDK, you define these three components in code:

  • Model: Strands offers flexible model support. You can use any model in Amazon Bedrock that supports tool use and streaming, a model from Anthropic's Claude model family through the Anthropic API, a model from the Llama model family via Llama API, Ollama for local development, and many other model providers such as OpenAI through LiteLLM. You can additionally define your own custom model provider with Strands.
  • Tools: You can choose from thousands of published Model Context Protocol (MCP) servers to use as tools for your agent. Strands also provides 20+ pre-built example tools, including tools for manipulating files, making API requests, and interacting with AWS APIs. You can easily use any Python function as a tool, by simply using the Strands @tool decorator.
  • Prompt: You provide a natural language prompt that defines the task for your agent, such as answering a question from an end user. You can also provide a system prompt that provides general instructions and desired behavior for the agent.

An agent interacts with its model and tools in a loop until it completes the task provided by the prompt. This agentic loop is at the core of Strands' capabilities. The Strands agentic loop takes full advantage of how powerful LLMs have become and how well they can natively reason, plan, and select tools. In each loop, Strands invokes the LLM with the prompt and agent context, along with a description of your agent's tools. The LLM can choose to respond in natural language for the agent's end user, plan out a series of steps, reflect on the agent's previous steps, and/or select one or more tools to use. When the LLM selects a tool, Strands takes care of executing the tool and providing the result back to the LLM. When the LLM completes its task, Strands returns the agent's final result.

In Strands' model-driven approach, tools are key to how you customize the behavior of your agents. For example, tools can retrieve relevant documents from a knowledge base, call APIs, run Python logic, or just simply return a static string that contains additional model instructions. Tools also help you achieve complex use cases in a model-driven approach, such as with these Strands Agents example pre-built tools:

  • Retrieve tool: This tool implements semantic search using Amazon Bedrock Knowledge Bases. Beyond retrieving documents, the retrieve tool can also help the model plan and reason by retrieving other tools using semantic search. For example, one internal agent at AWS has over 6,000 tools to select from! Models today aren't capable of accurately selecting from quite that many tools. Instead of describing all 6,000 tools to the model, the agent uses semantic search to find the most relevant tools for the current task and describes only those tools to the model. You can implement this pattern by storing many tool descriptions in a knowledge base and letting the model use the retrieve tool to retrieve a subset of relevant tools for the current task.
  • Thinking tool: This tool prompts the model to do deep analytical thinking through multiple cycles, enabling sophisticated thought processing and self-reflection as part of the agent. In the model-driven approach, modeling thinking as a tool enables the model to reason about if and when a task needs deep analysis.
  • Multi-agent tools like the workflow, graph, and swarm tools: For complex tasks, Strands can orchestrate across multiple agents in a variety of multi-agent collaboration patterns. By modeling sub-agents and multi-agent collaboration as tools, the model-driven approach enables the model to reason about if and when a task requires a defined workflow, graph, or swarm of sub-agents. Strands support for the Agent2Agent (A2A) protocol for multi-agent applications is coming soon.

Get started with Strands Agents

Let's walk through an example of building an agent with the Strands Agents SDK. As has long been said, naming things is one of the hardest problems in computer science. Naming an open source project is no exception! To help us brainstorm potential names for the Strands Agents project, I built a naming AI assistant using Strands. In this example, you will use Strands to build a naming agent using a default model in Amazon Bedrock, an MCP server, and a pre-built Strands tool.

Create a file named agent.py with this code:

python
from strands import Agent
from strands.tools.mcp import MCPClient
from strands_tools import http_request
from mcp import stdio_client, StdioServerParameters

# Define a naming-focused system prompt
NAMING_SYSTEM_PROMPT = """
You are an assistant that helps to name open source projects.

When providing open source project name suggestions, always provide
one or more available domain names and one or more available GitHub
organization names that could be used for the project.

Before providing your suggestions, use your tools to validate
that the domain names are not already registered and that the GitHub
organization names are not already used.
"""

# Load an MCP server that can determine if a domain name is available
domain_name_tools = MCPClient(lambda: stdio_client(
    StdioServerParameters(command="uvx", args=["fastdomaincheck-mcp-server"])
))

# Use a pre-built Strands Agents tool that can make requests to GitHub
# to determine if a GitHub organization name is available
github_tools = [http_request]

with domain_name_tools:
    # Define the naming agent with tools and a system prompt
    tools = domain_name_tools.list_tools_sync() + github_tools
    naming_agent = Agent(
        system_prompt=NAMING_SYSTEM_PROMPT,
        tools=tools
    )

    # Run the naming agent with the end user's prompt
    naming_agent("I need to name an open source project for building AI agents.")

You will need a GitHub personal access token to run the agent. Set the environment variable GITHUB_TOKEN with the value of your GitHub token. You will also need Bedrock model access for Anthropic Claude 3.7 Sonnet in us-west-2, and AWS credentials configured locally.

Now run your agent:

bash
pip install strands-agents strands-agents-tools
python -u agent.py

You should see output from the agent similar to this snippet:

text
Based on my checks, here are some name suggestions for your
open source AI agent building project:

## Project Name Suggestions:

1. **Strands Agents**
- Available domain: strandsagents.com
- Available GitHub organization: strands-agents

You can easily start building new agents today with the Strands Agents SDK in your favorite AI-assisted development tool. To help you quickly get started, we published a Strands MCP server to use with any MCP-enabled development tool, such as the Q Developer CLI or Cline. For the Q Developer CLI, use the following example to add the Strands MCP server to the CLI's MCP configuration. You can see more configuration examples on GitHub.

json
{
  "mcpServers": {
    "strands": {
      "command": "uvx",
      "args": ["strands-agents-mcp-server"]
    }
  }
}

Deploy Strands Agents in production

Running agents in production is a key tenet for the design of Strands. The Strands Agents project includes a deployment toolkit with a set of reference implementations to help you take your agents to production. Strands is flexible enough to support a variety of architectures in production. You can use Strands to build conversational agents as well as agents that are triggered by events, run on a schedule, or run continuously. You can deploy an agent built with the Strands Agents SDK as a monolith, where both the agentic loop and the tool execution run in the same environment, or as a set of microservices. I will describe four agent architectures that we use internally at AWS with Strands Agents.

The following diagram shows an agent architecture with Strands running entirely locally in a user's environment through a client application. The example command line tool on GitHub follows this architecture for a CLI-based AI assistant for building agents.

The next diagram shows an architecture where the agent and its tools are deployed behind an API in production. We have provided reference implementations on GitHub for how to deploy agents built with the Strands Agents SDK behind an API on AWS, using AWS Lambda, AWS Fargate, or Amazon Elastic Compute Cloud (Amazon EC2).

You can separate concerns between the Strands agentic loop and tool execution by running them in separate environments. The following diagram shows an agent architecture with Strands where the agent invokes its tools via API, and the tools run in an isolated backend environment separate from the agent's environment. For example, you could run your agent's tools in Lambda functions, while running the agent itself in a Fargate container.

You can also implement a return-of-control pattern with Strands, where the client is responsible for running tools. This diagram shows an agent architecture where an agent built with the Strands Agents SDK can use a mix of tools that are hosted in a backend environment and tools that run locally through a client application that invokes the agent.

Regardless of your exact architecture, observability of your agents is important for understanding how your agents are performing in production. Strands provides instrumentation for collecting agent trajectories and metrics from production agents. Strands uses OpenTelemetry (OTEL) to emit telemetry data to any OTEL-compatible backend for visualization, troubleshooting, and evaluation. Strands' support for distributed tracing enables you to track requests through different components in your architecture, in order to paint a complete picture of agent sessions.

Join the Strands Agents community

Strands Agents is an open source project licensed under the Apache License 2.0. We are excited to now build Strands in the open with you. We welcome contributions to the project, including adding support for additional providers' models and tools, collaborating on new features, or expanding the documentation. If you find a bug, have a suggestion, or have something to contribute, join us on GitHub.

To learn more about Strands Agents and to start building your own AI agents, check out the Strands Agents documentation and examples.

Explore Related Technologies & Topics

View All 9,111+ Repositories →

Discover similar high-velocity repositories, agent skills, and OpenAPI specifications across the ecosystem.

#apify Browse apify catalog → #automation Browse automation catalog → #crawler Browse crawler catalog → #crawling Browse crawling catalog → #headless Browse headless catalog → #headless-chrome Browse headless-chrome catalog → ⚡ MCP Servers Model Context Protocol → 🧠 Agent Skills Prompt & rule files → 🔌 OpenAPI Specs REST & Swagger schemas →