To Scale or Not To Scale?

Architecture Diagram
  1. Store API tokens from Interactive Brokers, IEXCloud, and other financial data platforms securely using AWS Secrets Manager.
  2. Use Amazon API Gateway to create an endpoint for our AWS Lambda functions and connect to these APIs securely using the API tokens.
  3. Use Lambda functions to access database credentials from Secrets Manager to connect
  4. Automatically rotate API keys and credentials every 30 days for best practice.
  5. Post real-time pricing and ticker data, using Lambda, to our internal PostgreSQL database from the endpoint, created using Amazon RDS.
  6. Execute trade orders against the database and query results using Lambda functions and store transaction data in real-time into a data lake, using Amazon Simple Storage Service (S3).
  7. Query data from the database using Lambda functions and then use AWS SageMaker to build, train, and deploy data science models based on findings.
  8. Query data from logs stored in S3 to feed data models in SageMaker.
  9. Send portfolio status reports to end users using Amazon Simple Notification Service (SNS) as the primary messaging service. (Add SNS to diagram and renumber steps)
  1. Operational Excellence
  2. Security
  3. Reliability
  4. Performance Efficiency
  5. Cost Optimization

Love podcasts or audiobooks? Learn on the go with our new app.

Recommended from Medium

Multi-Region Apache Cassandra on Azure Kubernetes Service with the K8ssandra Operator (Part One)

Leadership & organizational advice from Elon Musk

After Elon Musk gave his presentation about the SpaceX’s Starship program, Tim had the opportunity to pick his brain.

Basic Concepts of Selenium

Makers Pre-Course — Week 2

Ownership: Managing Memory in Rust

Solving “Bad Horse” — Google Kickstart 2013

GitHub: Create “” file like professional.

AWS IoT Core for LoRaWAN, Amazon IoT Analytics, and Amazon QuickSight

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store
Synergy Insights

Synergy Insights

More from Medium

A Key Problem in Deployment

Meet Mario López, Data Engineer at Mercadona Tech

Three case studies on API-first Development and Consumer-Driven Contracts