Original series by @lutendointech on TikTok — card format and content adapted here as a linkable reference. Notes compiled by the VeehiveLabs team. Corrections welcome.
Engineering Primer
8 infrastructure primitives every AI system builder should know
A one-page reference for the eight foundational systems we reach for on almost every enterprise AI build. Skim it, screenshot it, or point a junior engineer at it before their first architecture review. Not exhaustive — opinionated.
Kafka
Event StreamingWhat is it?
An event streaming platform that moves data between systems in real time.
Why was it created?
Applications generate millions of events that need to be processed reliably.
Where is it used?
- Order processing
- Activity tracking
- Analytics pipelines
- Notifications
Why should you care?
Instead of systems talking directly to each other, Kafka acts as a durable event pipeline — producers write once, many consumers read, and nothing gets lost when a downstream service goes down.
Nginx
Web Server · ProxyWhat is it?
A web server and reverse proxy.
Why was it created?
Handling thousands of simultaneous connections efficiently — something traditional threaded servers struggled with.
Where is it used?
- Load balancing
- SSL termination
- Static file delivery
- Reverse proxying
Why should you care?
Nginx sits at the front door of most production web apps: it handles traffic efficiently and routes requests to the right application servers behind it.
GraphQL
API Query LanguageWhat is it?
A query language for APIs.
Why was it created?
REST APIs often send too much or too little data — a mobile screen needing three fields would receive 30, or would trigger four round-trips.
Where is it used?
- Mobile apps
- Dashboards
- Complex frontends
Why should you care?
Clients can request exactly the data they need in a single request — often from multiple sources behind one endpoint. No over-fetching, no waterfall calls.
Elasticsearch
Distributed SearchWhat is it?
A distributed search engine.
Why was it created?
Database searches become slow on large datasets — especially fuzzy, full-text, or ranked queries that a SQL LIKE clause can’t handle at scale.
Where is it used?
- Website search
- Product search
- Log analysis
- Monitoring
Why should you care?
Optimised for fast full-text search across millions of records — with autocomplete, filters, and aggregations built in. Part of the ELK stack for log analysis.
Kubernetes
Container OrchestrationWhat is it?
A container orchestration platform.
Why was it created?
Managing hundreds of servers manually doesn’t scale — especially when you want auto-scaling, self-healing, and rolling updates.
Where is it used?
- Microservices
- Cloud infrastructure
- Automated deployments
- Scaling workloads
Why should you care?
Automates deployment, scaling, and recovery of applications — the control plane decides where things run so you can stop babysitting servers.
Redis
In-Memory StoreWhat is it?
An in-memory data store commonly used for caching and fast data access.
Why was it created?
Traditional databases can become slow when the same data is requested repeatedly — and disk reads are orders of magnitude slower than memory.
Where is it used?
- User sessions
- API caching
- Rate limiting
- Leaderboards
Why should you care?
A simple Redis cache can reduce response times from hundreds of milliseconds to just a few. It’s the fastest, cheapest performance win in most stacks.
RabbitMQ
Message BrokerWhat is it?
A message broker.
Why was it created?
Some tasks shouldn’t block user requests — sending an email or generating a PDF shouldn’t make an API call wait for seconds.
Where is it used?
- Emails
- Notifications
- Background processing
- Payment workflows
Why should you care?
Applications stay responsive while heavy work happens in the background. Producers publish once, exchanges route to queues, consumers pick up work asynchronously.
Docker
ContainersWhat is it?
A tool that packages applications and their dependencies into containers.
Why was it created?
Software often works differently across machines — the “works on my laptop” problem eating whole afternoons.
Where is it used?
- Development
- Testing
- Deployment
- CI/CD
Why should you care?
A Docker container behaves the same way on every machine — isolated, portable, and identical from your laptop to production. This is what makes Kubernetes possible.