Backend Weekly

API and API Design: API Security (part 2)

Solomom Eseme — Fri, 27 Jun 2025 07:20:30 +0000

Hello “👋

Welcome to another week, another opportunity to become a Great Backend Engineer.

Today’s issue is brought to you by Masteringbackend → A great resource for backend engineers. Learn backend development practically.

Still stuck in tutorial hell?

We’ve all been there—jumping from one Python video to the next, but never building anything real.
No portfolio. No confidence. No callbacks.

That’s why I created this.

The “Land Your Dream Python Job” Challenge
A 90-day, 3-phase roadmap that helps you:

✅ Build 30 real-world backend projects in 30 days
✅ Master DSA for technical interviews
✅ Get job-ready with resumes, mock interviews & daily job alerts
✅ And finally... land that backend job

This is NOT another course. It’s a challenge. And it works.

Over 2,000 Python developers have taken this path—many are now working at top companies.

Only 120 slots left at $54 (then goes up to $100)

Join the challenge & change your future
👉 python30.masteringbackend.com

Let’s get you unstuck.

Looking for unbiased, fact-based news? Join 1440 today.

Join over 4 million Americans who start their day with 1440 – your daily digest for unbiased, fact-centric news. From politics to sports, we cover it all by analyzing over 100 sources. Our concise, 5-minute read lands in your inbox each morning at no cost. Experience news without the noise; let 1440 help you make up your own mind. Sign up now and invite your friends and family to be part of the informed.

Subscribe to 1440 today.

Welcome to another episode of Backend Weekly, previously, I covered the top 4 security risks from OWASP Top 10, and in this issue, I will continue with API Security and cover the remaining 6 API security tips from the OWASP 10 list below:

✅ Broken Authentication
✅ Broken Object Level Authorization
✅ Broken Object Property Level Authorization
✅ Unrestricted Resource Consumption

Next, I will cover the following:

Broken Function Level Authorization
Unrestricted Access to Sensitive Business Flows
Server-Side Request Forgery
Security Misconfiguration
Improper Inventory Management
Unsafe Consumption of APIs

Broken Function Level Authorization

This issue is listed in the OWASP API Security Top 10 as API5 – Broken Function Level Authorization.

It occurs when APIs expose endpoints that don’t properly check whether the authenticated user has permission to access the function being called.

Complex access control policies—such as those involving user roles, groups, and permissions—can lead to improper checks or lack of authorization logic altogether.

Example:

Below is an example that covers the broken function level authorization security bridge and some of the possible solutions. Suppose an API has both regular users and admin users, but it fails to enforce role-based authorization:

// INSECURE: No role check — anyone can access admin route
app.delete('/admin/delete-user/:id', async (req, res) => {
  await db.User.deleteOne({ _id: req.params.id });
  res.send('User deleted');
});

Solution:

Implement middleware to enforce role-based access control (RBAC):

function requireAdmin(req, res, next) {
  if (req.user.role !== 'admin') {
    return res.status(403).send('Access denied');
  }
  next();
}

app.delete('/admin/delete-user/:id', requireAdmin, async (req, res) => {
  await db.User.deleteOne({ _id: req.params.id });
  res.send('User deleted');
});

Additionally, ensure that roles are properly validated during login and token creation:

const token = jwt.sign({ id: user.id, role: user.role }, process.env.JWT_SECRET);

By adding a clear separation of duties and verifying access based on user role or group, APIs can better prevent unauthorized access to sensitive operations.

Unrestricted Access to Sensitive Business Flows

This vulnerability is defined in the OWASP API Security Top 10 (2023) as API6 – Unrestricted Access to Sensitive Business Flows. It arises not from code bugs, but from a lack of strategic business rule enforcement.

APIs may expose critical operations—like purchasing items, creating accounts, or submitting comments—without considering how these operations could be abused at scale. This allows attackers or bots to overwhelm business services in ways that hurt revenue, credibility, or functionality.

Example:

Suppose you have an endpoint that allows users to post comments on a product:

// INSECURE: No user limits, abuse protection, or flow control
app.post('/products/:id/comment', async (req, res) => {
  const { comment } = req.body;
  const { id } = req.params;
  await db.Comment.create({ productId: id, userId: req.user.id, text: comment });
  res.send('Comment added');
});

A spam bot could hit this endpoint thousands of times, flooding your app with spam and damaging trust with legitimate users.

Solution:

Introduce safeguards including:

Rate limiting to restrict frequency.
Behavioral analysis to detect bots.
Business logic rules to cap use (e.g., one comment per user per 10 minutes).

const rateLimit = require('express-rate-limit');

const commentLimiter = rateLimit({
  windowMs: 10 * 60 * 1000, // 10 minutes
  max: 3, // Max 3 comments per window
  message: 'Commenting too frequently. Please slow down.'
});

app.post('/products/:id/comment', commentLimiter, async (req, res) => {
  const { comment } = req.body;
  const { id } = req.params;

  const recent = await db.Comment.find({
    productId: id,
    userId: req.user.id,
    createdAt: { $gte: new Date(Date.now() - 10 * 60 * 1000) }
  });

  if (recent.length >= 1) {
    return res.status(429).send('You can only comment once every 10 minutes.');
  }

  await db.Comment.create({ productId: id, userId: req.user.id, text: comment });
  res.send('Comment added');
});

This ensures sensitive flows are governed by rate and relevance, reducing the risk of economic abuse or reputational harm.

Server-Side Request Forgery

This vulnerability is listed under OWASP API Security Top 10: API7 – Server Side Request Forgery.

SSRF vulnerabilities occur when an API endpoint allows users to specify URLs or IP addresses to fetch remote resources, but the server does not adequately validate these inputs.

Attackers exploit this to make the server issue requests to internal systems, cloud metadata endpoints, or other protected environments.

Example:

Here's an example of a vulnerable Node.js/Express endpoint that accepts a URL as input:

// INSECURE: No input validation, direct use of user-supplied URL
const axios = require('axios');

app.get('/proxy', async (req, res) => {
  const { url } = req.query;
  try {
    const response = await axios.get(url);
    res.send(response.data);
  } catch (err) {
    res.status(500).send('Error fetching resource');
  }
});

With this implementation, a malicious actor can make the server request internal URLs such as:

GET /proxy?url=http://localhost:8000/admin
GET /proxy?url=http://169.254.169.254/latest/meta-data/  // AWS metadata

This can lead to data exfiltration or allow attackers to bypass firewall protections.

Solution:

Implement input validation to allow only approved domains.
Use allowlists to define safe URLs or domains.
Disallow local IPs like 127.0.0.1, 169.254.x.x, and internal ranges.

const dns = require('dns');
const isValidUrl = require('valid-url');

app.get('/proxy', async (req, res) => {
  const { url } = req.query;

  if (!isValidUrl.isUri(url)) {
    return res.status(400).send('Invalid URL');
  }

  const parsedUrl = new URL(url);
  const hostname = parsedUrl.hostname;

  // Only allow requests to a predefined list of domains
  const allowedHosts = ['example.com', 'api.trusted.com'];
  if (!allowedHosts.includes(hostname)) {
    return res.status(403).send('Access denied');
  }

  try {
    const response = await axios.get(url);
    res.send(response.data);
  } catch (err) {
    res.status(500).send('Error fetching resource');
  }
});

In highly sensitive applications, use an external proxy layer to isolate request fetching from your API server entirely.

APIs are essential to the architecture of modern software systems, but they also introduce new security risks. API security must be integrated from the earliest stages of design and remain a continuous practice throughout development and operation.

As APIs grow in number and complexity, so do the threats. Prioritize security, implement best practices, and choose the right tools to safeguard your API ecosystem.

Did you learn any new things from this newsletter this week? Please reply to this email and let me know. Feedback like this encourages me to keep going.

Remember to start learning backend engineering from our courses:

Get a 50% discount on any of these courses. Reach out to me (Reply to this mail)

Backend Engineering Resources

Whenever you're ready

There are 4 ways I can help you become a great backend engineer:

1. The MB Platform: Join 1000+ backend engineers learning backend engineering on the MB platform. Build real-world backend projects, track your learnings and set schedules, learn from expert-vetted courses and roadmaps, and solve backend engineering tasks, exercises, and challenges.

2. The MB Academy: The “MB Academy” is a 6-month intensive Advanced Backend Engineering BootCamp to produce great backend engineers.

3. MB Video-Based Courses: Join 1000+ backend engineers who learn from our meticulously crafted courses designed to empower you with the knowledge and skills you need to excel in backend development.

4. GetBackendJobs: Access 1000+ tailored backend engineering jobs, manage and track all your job applications, create a job streak, and never miss applying. Lastly, you can hire backend engineers anywhere in the world.

LAST WORD 👋

How am I doing?

I love hearing from readers, and I'm always looking for feedback. How am I doing with The Backend Weekly? Is there anything you'd like to see more or less of? Which aspects of the newsletter do you enjoy the most?

Hit reply and say hello - I'd love to hear from you!

Stay awesome,
Solomon

I moved my newsletter from Substack to Beehiiv, and it's been an amazing journey. Start yours here.

API and API Design: API Security

Solomom Eseme — Mon, 16 Jun 2025 12:01:10 +0000

Hello “👋

Welcome to another week, another opportunity to become a Great Backend Engineer.

Today’s issue is brought to you by Masteringbackend → A great resource for backend engineers. Learn backend development practically.

Before we get started, I have a few announcements:

I have a special gift for you: You will love this one.

The ultimate “Land Your Dream Job” Challenge is here.

We are launching the ultimate guide to landing your dream job in any programming language you choose. We are starting with the Python Programming language.

Land your dream Python Job in 90 days by shipping 30 Python projects in 30 days by completing our daily tasks.

It’s a cohort-based and project-focused challenge where you will be challenged to build 30 Python projects in 30 days.

Here is what you will get:

Ship 30+ Python backend projects in 30 days.
Instant Access to all 30+ videos
Access to data structure and algorithm interview kits.
Access our Complete Backend Job Preparation kits (Resume, Cover letter reviews, mock interviews, and job placements).
Join & learn from a thriving community of helpful students & alumni from top companies.

Limited Access. The first 500 students will be at $54, others at $100 (We have only 120 slots left.)

Click here to join the challenge.

Introduction

Securing an API is not an afterthought—it is a fundamental aspect of the API design lifecycle.

As APIs increasingly serve as the connective tissue of modern applications, their exposure to the internet makes them attractive targets for cyberattacks.

Proper API security ensures that an API reliably fulfills its intended purpose while safeguarding sensitive data, maintaining service availability, and upholding user trust.

API security is a set of strategies, practices, and technologies designed to protect APIs from unauthorized access, abuse, data breaches, and performance degradation.

What is API Security?

API security encompasses the practices and tools used to secure APIs throughout their entire lifecycle. This includes measures to authenticate users, authorize access, validate inputs, encrypt data, limit usage, monitor behavior, and handle threats.

APIs expose business logic and data to a variety of clients—including browsers, mobile apps, third-party services, and IoT devices. Because of this broad exposure, any weakness in API design can become an attack vector.

Below are some of the core objectives of API Security:

Core Objectives of API Security

Prevent unauthorized access to sensitive resources.
Protect against data leaks and injection attacks.
Ensure high availability and mitigate DoS attacks.
Preserve data integrity and privacy.

Next, let’s explore some of the common API Security Threats according to OWASP Top 10 API Security report.

Common API Security Threats

According to OWASP Top 10 API Security 2023, there are top 10 web and API security vulnerabilities that should be implemented on every API by API developers, and below is the list of them.

Broken Authentication
Broken Object Level Authorization
Broken Object Property Level Authorization
Unrestricted Resource Consumption
Broken Function Level Authorization
Unrestricted Access to Sensitive Business Flows
Server-Side Request Forgery
Security Misconfiguration
Improper Inventory Management
Unsafe Consumption of APIs

Broken Authentication

APIs that fail to correctly validate tokens or credentials may allow unauthorized access. For example, consider a Node.js API using JWT for authentication but failing to verify the token properly:

// INSECURE: Token is extracted but not verified
app.get('/secure-data', (req, res) => {
  const token = req.headers['authorization'];
  const decoded = jwt.decode(token); // decode() does not verify signature
  if (!decoded) return res.status(401).send('Unauthorized');
  res.send('Sensitive data');
});

In this scenario, any user could tamper with or forge a token and gain access to protected routes because the server does not verify the signature of the JWT.

Solution:

Use jwt.verify() to ensure the token's integrity and authenticity:

const jwt = require('jsonwebtoken');

app.get('/secure-data', (req, res) => {
  const token = req.headers['authorization']?.split(' ')[1];
  if (!token) return res.status(401).send('Token missing');

  try {
    const decoded = jwt.verify(token, process.env.JWT_SECRET);
    req.user = decoded;
    res.send('Sensitive data');
  } catch (err) {
    res.status(403).send('Invalid token');
  }
});

This ensures that the token was not tampered with and that only authenticated users can access protected endpoints.

Excessive Data Exposure

Sending too much data in API responses can reveal sensitive or internal information, especially if the backend inadvertently sends entire database records or fields that should be hidden. This issue is covered under OWASP API Security Top 10: API3 – Excessive Data Exposure.

Consider this insecure Node.js/Express example:

// INSECURE: returns all user fields including sensitive data
app.get('/user/:id', async (req, res) => {
  const user = await db.User.findById(req.params.id);
  res.json(user); // Sends password hash, tokens, internal flags, etc.
});

In this code, sensitive information such as password hashes, tokens, or internal flags might be leaked in the API response.

Solution:

Use selective field projection or response filtering to return only what’s needed:

// SECURE: Sends only public-facing user fields
app.get('/user/:id', async (req, res) => {
  const user = await db.User.findById(req.params.id);
  if (!user) return res.status(404).send('User not found');

  const safeResponse = {
    id: user.id,
    name: user.name,
    email: user.email
  };

  res.json(safeResponse);
});

Alternatively, many ORMs like Mongoose offer built-in projection:

const user = await User.findById(req.params.id).select('id name email');

By explicitly defining the fields returned in API responses, developers can avoid leaking sensitive internal data and ensure better adherence to the principle of least privilege.

Lack of Rate Limiting

Without usage limits, APIs are vulnerable to brute-force attacks and abuse. Attackers can exploit endpoints by sending repeated requests to guess credentials, flood the system with traffic, or scrape data uncontrollably.

Here’s a Node.js/Express example that lacks rate limiting:

// INSECURE: No rate limiting middleware
app.post('/login', async (req, res) => {
  const { username, password } = req.body;
  const user = await db.User.findOne({ username });
  if (!user || user.password !== password) return res.status(401).send('Invalid credentials');
  res.send('Logged in');
});

In this case, a malicious actor could automate requests to brute-force user credentials, overwhelming the system or gaining unauthorized access.

Solution:

Implement rate-limiting middleware using express-rate-limit

const rateLimit = require('express-rate-limit');

const loginLimiter = rateLimit({
  windowMs: 15 * 60 * 1000, // 15 minutes
  max: 10, // Limit each IP to 10 login requests per windowMs
  message: 'Too many login attempts from this IP, please try again later.'
});

app.post('/login', loginLimiter, async (req, res) => {
  const { username, password } = req.body;
  const user = await db.User.findOne({ username });
  if (!user || user.password !== password) return res.status(401).send('Invalid credentials');
  res.send('Logged in');
});

This effectively mitigates brute-force attempts and protects your authentication endpoint by throttling repeated access from the same IP address.

Injection Attacks

Improperly validated inputs may lead to SQL, command, or script injection. This vulnerability is listed as OWASP API Security Top 10: API8 – Injection.

Suppose we have a Node.js/Express API that takes user input and constructs a raw SQL query without any sanitization:

// INSECURE: Raw SQL query vulnerable to injection
app.get('/search', async (req, res) => {
  const { term } = req.query;
  const results = await db.query(`SELECT * FROM products WHERE name LIKE '%${term}%'`);
  res.json(results);
});

If a user enters something like %' OR '1'='1, this could result in the query:

SELECT * FROM products WHERE name LIKE '%%' OR '1'='1%'

This would return all products, allowing attackers to extract unintended data.

Solution:

Use parameterized queries or an ORM that automatically escapes user input:

// SECURE: Parameterized query using query placeholders
app.get('/search', async (req, res) => {
  const { term } = req.query;
  const results = await db.query('SELECT * FROM products WHERE name LIKE ?', [`%${term}%`]);
  res.json(results);
});

Alternatively, if using an ORM like Sequelize:

const results = await Product.findAll({
  where: {
    name: {
      [Op.like]: `%${term}%`
    }
  }
});

Always validate and sanitize user input. Never directly inject raw input into database queries, command shells, or other interpreters.

That will be all for this newsletter issue. However, in the next one, we will cover the remaining 6 from the OWASP Top 10.

Key Principles of API Security

Authentication: Ensure that only verified users or services can access the API. Use robust mechanisms such as OAuth 2.0, API keys, or JWT-based auth.
Authorization: After authentication, verify what actions the user or client is permitted to perform. Implement role-based access control (RBAC) or attribute-based access control (ABAC).
Input Validation: Sanitize and validate all input to prevent injection attacks and ensure data integrity.
Rate Limiting & Throttling: Protect resources from abuse by capping the number of requests a client can make.
Data Encryption: Use HTTPS/TLS to encrypt data in transit. For highly sensitive data, consider encrypting at rest as well.
Logging & Monitoring: Track API access, failed logins, and anomalies. Use tools like Prometheus, Grafana, or ELK stack.
Versioning & Deprecation: Avoid breaking changes by versioning APIs. Remove unused endpoints to reduce attack surface

From authentication and authorization to input validation and traffic monitoring, a comprehensive API security strategy not only protects data and services—it also ensures trust, reliability, and compliance.

As APIs grow in number and complexity, so do the threats. Prioritize security, implement best practices, and choose the right tools to safeguard your API ecosystem.

Did you learn any new things from this newsletter this week? Please reply to this email and let me know. Feedback like this encourages me to keep going.

Remember to start learning backend engineering from our courses:

Get a 50% discount on any of these courses. Reach out to me (Reply to this mail)

Backend Engineering Resources

Whenever you're ready

There are 4 ways I can help you become a great backend engineer:

2. The MB Academy: The “MB Academy” is a 6-month intensive Advanced Backend Engineering BootCamp to produce great backend engineers.

LAST WORD 👋

How am I doing?

Hit reply and say hello - I'd love to hear from you!

Stay awesome,
Solomon

I moved my newsletter from Substack to Beehiiv, and it's been an amazing journey. Start yours here.

API & API Design: API Documentation Tools

Solomom Eseme — Wed, 30 Apr 2025 17:45:36 +0000

Hello “👋

Welcome to another week, another opportunity to become a Great Backend Engineer.

Today’s issue is brought to you by Masteringbackend → A great resource for backend engineers. We offer next-level backend engineering training and exclusive resources.

Before we get started, I have a few announcements:

I have a special gift for you: You will love this one.

The ultimate “Land Your Dream Job” Challenge is here.

We are launching the ultimate guide to landing your dream job in any programming language you choose. We are starting with the Python Programming language.

Land your dream Python Job in 90 days by shipping 30 Python projects in 30 days by completing our daily tasks.

It’s a cohort-based and project-focused challenge where you will be challenged to build 30 Python projects in 30 days.

Here is what you will get:

Ship 30+ Python backend projects in 30 days.
Instant Access to all 30+ videos
Access to data structure and algorithm interview kits.
Access our Complete Backend Job Preparation kits (Resume, Cover letter reviews, mock interviews, and job placements).
Join & learn from a thriving community of helpful students & alumni from top companies.

Limited Access. The first 500 students will be at $54, others at $100 (We have only 220 slots left.)

Click here to join the challenge.

You Don’t Need to Be Technical. Just Informed

AI isn’t optional anymore—but coding isn’t required.

The AI Report gives business leaders the edge with daily insights, use cases, and implementation guides across ops, sales, and strategy.

Trusted by professionals at Google, OpenAI, and Microsoft.

👉 Get the newsletter and make smarter AI decisions.

Lead with AI—no coding needed

Introduction

Documentation is important even in API and API Design.

Therefore, writing clear and comprehensive documentation is as critical as the API development process itself.

How many of you have experienced bad documentation for the API you want to use in your project? Do you remember how frustrating it was?

If you have experienced it before, you can agree with me that bad API documentation hurts the developer experience of your API. That is why you need to plan out your API documentation correctly.

This issue delves into the importance of API documentation tools, how they improve developer onboarding, and a comparison of popular tools such as Swagger (OpenAPI), DapperDox, Postman, and ReDoc.

Why API Documentation Matters

API documentation provides a reference point for developers on how to use the API effectively. Good documentation includes:

Endpoint definitions (methods, URLs, request/response formats)
Authentication requirements
Error codes and responses
Example requests and responses
Code samples in different languages

Without accurate and accessible documentation, even the most powerful API can become difficult to use. It’s the bridge between your backend logic and the developers consuming your services.

Next, if you’re looking at choosing and integrating a documentation tool into your API workflow. You need to understand what makes a good documentation tool and how each of these tools is different in its unique way.

What makes a Good Documentation Tool?

An ideal API documentation tool should:

Support auto-generation from code or API definitions (e.g., OpenAPI specs).
Offer a searchable, user-friendly interface.
Allow interactive testing (try-it-out functionality).
Be customizable and themeable.
Enable version control and change tracking.
Provide support for multi-language code samples.

Looking at these features, you see that I am only interested in the idea features that truly matter in a documentation tool and not the features of individual tools, because I want you to pick any tool of your choice, knowing exactly what you want in it.

ReDoc

ReDoc is another OpenAPI-powered tool focused on simplicity and beautiful, responsive documentation. Let’s explore some of the key features of ReDoc.

Fully customizable with CSS and branding
Supports deep linking and markdown
Performance-optimized for large APIs
Minimalist, clean UI ideal for production docs

ReDoc is a great open-source tool that is best for public-facing APIs where branding and UX matter.

Postman

Postman is a popular API platform known primarily for API testing, but it also offers a feature-rich documentation system that integrates tightly with collections. If you’ve been building APIs, you should already know Postman and some of the key features as listed below:

Auto-generates docs from Postman collections
Interactive documentation with embedded testing
Supports Markdown descriptions
Collaboration features and versioning
Public or private sharing of docs

Postman is a suite of API testing and management tools best for teams who want seamless documentation generation without writing OpenAPI manually.

How to Choose the Right Tool

Choosing the right documentation tool to streamline your API development can be difficult because of the numerous tools available. However, here are factors to consider:

How to Choose the Right Tool

API documentation tools are no longer optional; they are fundamental to delivering an exceptional developer experience. Whether you’re a solo backend engineer or a large API-first team, if you can choose the right documentation tool, it will help your API speak clearly and confidently to the world.

Just to add:

If you want to go deeper into Documentation Engineering, I have created a complete roadmap course that is focused on engineers like you and walks you through how to become a great documentation engineer.

Check it out here.

Did you learn any new things from this newsletter this week? Please reply to this email and let me know. Feedback like this encourages me to keep going.

Remember to start learning backend engineering from our courses:

Get a 50% discount on any of these courses. Reach out to me (Reply to this mail)

Backend Engineering Resources

Whenever you're ready

There are 4 ways I can help you become a great backend engineer:

2. The MB Academy: The “MB Academy” is a 6-month intensive Advanced Backend Engineering BootCamp to produce great backend engineers.

LAST WORD 👋

How am I doing?

Hit reply and say hello - I'd love to hear from you!

Stay awesome,
Solomon

I moved my newsletter from Substack to Beehiiv, and it's been an amazing journey. Start yours here.

API and API Design: API Keys & Management

Solomom Eseme — Tue, 29 Apr 2025 19:05:01 +0000

Hello “👋

Welcome to another week, another opportunity to become a Great Backend Engineer.

Today’s issue is brought to you by Masteringbackend → A great resource for backend engineers. We offer next-level backend engineering training and exclusive resources.

Before we get started, I have a few announcements:

I have a special gift for you: You will love this one.

The ultimate “Land Your Dream Job” Challenge is here.

We are launching the ultimate guide to landing your dream job in any programming language you choose. We are starting with the Python Programming language.

Land your dream Python Job in 90 days by shipping 30 Python projects in 30 days by completing our daily tasks.

It’s a cohort-based and project-focused challenge where you will be challenged to build 30 Python projects in 30 days.

Here is what you will get:

Ship 30+ Python backend projects in 30 days.
Instant Access to all 30+ videos
Access to data structure and algorithm interview kits.
Access our Complete Backend Job Preparation kits (Resume, Cover letter reviews, mock interviews, and job placements).
Join & learn from a thriving community of helpful students & alumni from top companies.

Limited Access. The first 500 students will be at $54, others at $100 (We have only 220 slots left.)

Click here to join the challenge.

Start learning AI in 2025

Keeping up with AI is hard – we get it!

That’s why over 1M professionals read Superhuman AI to stay ahead.

Get daily AI news, tools, and tutorials
Learn new AI skills you can use at work in 3 mins a day
Become 10X more productive

Introduction

APIs form the backbone of data exchange and service integration across web, mobile, and cloud-based systems. However, it is paramount to ensure that APIs are accessed securely and responsibly.

Therefore, to ensure that this is achieved, there are two central elements to understand, viz:

API Keys
API Management

In today’s issue, we will explore the concept of API Keys, how they function, and the role of API Management in maintaining security and performance in API design.

What is an API Key?

An API key is a unique string generated by the API provider to authenticate and identify the calling program, developer, or application requesting access to the API. It acts as a simple form of authentication and is typically included in request headers, query strings, or body payloads.

GET /api/v1/weather?city=London
Host: api.example.com
x-api-key: abc123xyz456

The snippet above passes a simple x-api-key to the request. API Keys are simple to generate and use, uniquely associated with users or applications, used to monitor and control usage (e.g., rate limits, quotas), and often the first layer of security before deeper authentication mechanisms.

If you’ve built an API before, you may have requested that your frontend engineers always pass either a JWT or a normal token to validate an authenticated user. Now, that is an example of one kind of API Key that could be passed.

Next, let’s explore some benefits and limitations of API keys in API design:

Benefits of API Keys

Despite being a basic method, API keys provide several essential advantages:

Access Control: You can issue unique API keys per client and revoke or regenerate them if needed.
Rate Limiting & Quotas: Each key can be tied to specific usage limits, preventing service abuse or overuse.
Monitoring & Analytics: Issuing individual keys to specific users can help track usage patterns, popular endpoints, and detect anomalies on a per-key basis.
Simplicity: No complex authentication handshake—making API keys great for internal systems or simple integrations.

Limitations of API Keys

Sometimes, API keys can also have significant security limitations:

They do not verify the identity of the end user.
API keys are static and, if leaked, can be easily misused.
They are often transmitted in plaintext unless HTTPS is enforced.

Therefore, while useful, API keys are often combined with more secure methods such as OAuth or JWT in production-grade systems.

Moving on, let’s look at API Management to understand the set of practices used to design and scale APIs.

What is API Management?

API Management refers to the comprehensive set of practices, policies, and tools that organizations use to design, deploy, secure, monitor, and scale APIs throughout their lifecycle.

Below are some of the key responsibilities of API Management:

Design & Development: This stage involves the design and development of the API, and tools such as Swagger or Postman help in designing APIs with standardized specifications.
Security Enforcement: API Management involves managing authentication (API Keys, OAuth, JWT), encryption (HTTPS), IP whitelisting, and rate limiting.
Monitoring and Analytics: This involves capturing API usage statistics, error rates, latency, and client behavior.
Versioning & Lifecycle Management: Handling changes in APIs (e.g., v1 to v2), deprecations, backward compatibility, and changelogs.
Developer Experience: Offering developer portals with documentation, testing consoles, key management, and sandbox environments.

Everything mentioned above and more make up the API Management. Here’s a simplified diagram showing how API keys integrate within the broader context of API management:

API Management Tools

Several platforms provide comprehensive API management capabilities, and below are the common platforms you can check out.

Tools	Features
Kong	Open-source gateway, plugins for rate limiting, logging
Apigee (Google)	Enterprise-grade, full lifecycle management
AWS API Gateway	Integration with AWS IAM, Lambda, and monitoring tools
Postman	Primarily for design/testing, but includes key generation
Azure API Management	Scalable, secure, integrates with Azure AD, and provides insights

Furthermore, let’s look at some of the best practices you can implement when building out your API Keys and Management pipeline.

Always use HTTPS to prevent key exposure over the network.
Rotate keys regularly to reduce long-term exposure.
Limit permissions per key (least privilege principle).
Use environment-based keys (e.g., different keys for dev, test, prod).
Implement IP restrictions to limit access from specific sources.
Monitor and alert on unusual API usage patterns.

API keys offer a lightweight mechanism for securing access to APIs, but they’re best suited for low-risk or internal use cases. For robust security, they should be part of a broader API management strategy that includes access control, monitoring, rate limiting, and lifecycle governance.

Rate limiting per API Key

API management platforms empower organizations to deliver high-quality APIs with agility, while keeping access secure, usage optimized, and developers productive.

As API ecosystems grow in complexity and scale, mastering both API key usage and API management practices becomes essential for backend engineers, developers, or architects designing modern software solutions.

Did you learn any new things from this newsletter this week? Please reply to this email and let me know. Feedback like this encourages me to keep going.

Remember to start learning backend engineering from our courses:

Get a 50% discount on any of these courses. Reach out to me (Reply to this mail)

Backend Engineering Resources

Whenever you're ready

There are 4 ways I can help you become a great backend engineer:

2. The MB Academy: The “MB Academy” is a 6-month intensive Advanced Backend Engineering BootCamp to produce great backend engineers.

LAST WORD 👋

How am I doing?

Hit reply and say hello - I'd love to hear from you!

Stay awesome,
Solomon

I moved my newsletter from Substack to Beehiiv, and it's been an amazing journey. Start yours here.

API & API Design: Authorization Methods

Solomom Eseme — Sun, 27 Apr 2025 12:12:00 +0000

Hello “👋

Welcome to another week, another opportunity to become a Great Backend Engineer.

Today’s issue is brought to you by Masteringbackend → A great resource for backend engineers. We offer next-level backend engineering training and exclusive resources.

Before we get started, I have a few announcements:

I have a special gift for you: You will love this one.

The ultimate “Land Your Dream Job” Challenge is here.

We are launching the ultimate guide to landing your dream job in any programming language you choose. We are starting with the Python Programming language.

Land your dream Python Job in 90 days by shipping 30 Python projects in 30 days by completing our daily tasks.

It’s a cohort-based and project-focused challenge where you will be challenged to build 30 Python projects in 30 days.

Here is what you will get:

Ship 30+ Python backend projects in 30 days.
Instant Access to all 30+ videos
Access to data structure and algorithm interview kits.
Access our Complete Backend Job Preparation kits (Resume, Cover letter reviews, mock interviews, and job placements).
Join & learn from a thriving community of helpful students & alumni from top companies.

Limited Access. The first 500 students will be at $54, others at $100 (We have only 220 slots left.)

Click here to join the challenge.

Find out why 1M+ professionals read Superhuman AI daily.

In 2 years you will be working for AI

Or an AI will be working for you

Here's how you can future-proof yourself:

Join the Superhuman AI newsletter – read by 1M+ people at top companies
Master AI tools, tutorials, and news in just 3 minutes a day
Become 10X more productive using AI

Join 1,000,000+ pros at companies like Google, Meta, and Amazon that are using AI to get ahead.

In our previous issue, "Authentication Methods in API Design," we explored the most widely adopted authentication methods in API design, detailing how they work, when to use them, and how to implement them securely.

This issue explores API design's most widely adopted authorization methods, detailing how they work, when to use them, and how to implement them securely.

Introduction

In API design, ensuring that authenticated users only access the resources they are permitted to access is just as important as verifying who they are. This is where authorization comes in.

While authentication answers the "Who are you?" authorization asks, "What are you allowed to do?"

Authorization methods form the backbone of secure access control in APIs. They enforce rules that determine which clients—whether users, services, or applications—can access what data and perform what actions. This is essential for protecting sensitive information, preserving data integrity, and ensuring business logic is correctly enforced.

This issue explores key authorization methods in API design, how they function, and when to use each based on security, scalability, and user experience considerations.

Authorization vs. Authentication

Before diving into the various methods, it’s important to distinguish authentication from authorization:

Authentication verifies who a user or application is.
Authorization verifies what they can access or do.

While often used together, they serve distinct purposes and require different implementation strategies.

Role-Based Access Control (RBAC)

RBAC assigns permissions to users based on their role within an organization. For example, a user with an "Admin" role may have access to more resources than a "Viewer".

Role-Based Access Control (RBAC)

Roles are defined at the system level, and each role has predefined privileges. When a user is authenticated, the API checks their assigned role and determines whether they are authorized to perform the requested action.

How it Works

Users are assigned roles (e.g., Admin, Editor, Viewer).
Each role is mapped to a set of permissions.
The API checks the user’s role on each request and determines whether the action is allowed.

For example, a customer support agent needs access to user account information but does not require administrative control over system settings, whereas a system administrator does.

All RBAC models include the following core elements:

Administrators: Users responsible for defining roles and assigning permissions.
Roles: Groups of users categorized by the tasks or responsibilities they perform.
Permissions: Each role is granted specific actions and access levels, which determine what users in that role are allowed to do.

Role-Based Access Control (RBAC) enables administrators to create roles, assign users to them, and manage permissions to control access within a system.

if (req.user.role !== 'admin') {
  return res.status(403).send('Forbidden: Admins only');
}

Role-based access control is best for Enterprise systems with well-defined roles and Multi-user platforms like CMSs or SaaS apps.

Attribute-Based Access Control (ABAC)

ABAC controls access using attributes rather than roles. These attributes can include user details (e.g., department, clearance level), resource metadata, or environmental conditions (e.g., time of day).

Attribute-Based Access Control (ABAC)

ABAC is highly flexible and supports complex access policies.

How it Works

Define attributes for users, resources, and environment.
Create policies based on combinations of these attributes.
Evaluate each request against the relevant policy.

if (req.user.department === 'finance' && req.resource.type === 'report') {
  // allow access
}

The attribute-based access control is best for fine-grained and dynamic access control and scenarios with many user-resource-environment combinations.

Conclusion

Properly implemented authorization is essential to any API's security architecture. Whether you’re designing internal APIs or exposing functionality to third parties, choosing the right authorization strategy ensures that only permitted actors can perform certain operations.

RBAC offers straightforward, role or resource-based permissions.
ABAC allows for highly dynamic and context-aware access policies.

As your application grows, so do your authorization needs. Consider starting with a simple RBAC model and evolving toward ABAC as complexity increases. Whichever method you choose, ensure it is consistently enforced across your API ecosystem.

Did you learn any new things from this newsletter this week? Please reply to this email and let me know. Feedback like this encourages me to keep going.

Remember to start learning backend engineering from our courses:

Get a 50% discount on any of these courses. Reach out to me (Reply to this mail)

Backend Engineering Resources

Whenever you're ready

There are 4 ways I can help you become a great backend engineer:

2. The MB Academy: The “MB Academy” is a 6-month intensive Advanced Backend Engineering BootCamp to produce great backend engineers.

LAST WORD 👋

How am I doing?

Hit reply and say hello - I'd love to hear from you!

Stay awesome,
Solomon

I moved my newsletter from Substack to Beehiiv, and it's been an amazing journey. Start yours here.

[New Podcast]: Rust + AWS Lambda: The Power of Serverless Performance

Solomom Eseme — Wed, 16 Apr 2025 09:30:57 +0000

Hello “👋

Welcome to another week, another opportunity to become a Great Backend Engineer.

Today’s issue is brought to you by Masteringbackend → A great resource for backend engineers. We offer next-level backend engineering training and exclusive resources.

Before we get started, I have a few announcements:

I have a special gift for you: You will love this one.

Easter Offer: 60% Off All Backend Engineering Courses

This Easter, we offer a rare opportunity to accelerate your backend engineering journey.

Easter Offer: 60% Off All Backend Engineering Courses

For a limited time only, you can access any course on Masteringbackend at 60% off. Whether you're just starting or looking to deepen your expertise, our curated backend engineering tracks are built to help you go beyond tutorials and build real-world skills.

What You’ll Get With Every Course:

Hands-on projects based on real-world backend scenarios
Comprehensive tracks for Python, Node.js, Rust, Java, and more
Complete backend job preparation kits
Mock interviews, resume reviews, and career support
A private community of backend engineers and mentors
Certificates of completion to validate your skills

Thousands of developers have used our platform to build real projects, master backend fundamentals, and land roles at top companies.

Now it’s your turn—at a fraction of the cost.

Limited Access. The promo code below is only for the first 20 people to use it. Secure your access at 60% off before the offer expires in 72 hours.

Visit the Course Catalog. Use “EASTER60”

What happens when you combine the speed and safety of Rust with the scalability of AWS Lambda?

I recently sat down with Luciano Mammino, Senior Architect and the brilliant mind behind the bestselling book on “Node.js Design Patterns,” to find out — and the result is one of the most insightful podcast episodes we’ve ever recorded.

Watch now → Rust + AWS Lambda: The Power of Serverless Performance

In this episode, we dive deep into:

Why Rust is a hidden gem for backend developers: Luciano explains how Rust’s performance, safety, and type system make it a top-tier language for serious backend work.
The magic of Rust + Lambda: Discover how smaller binaries and lightning-fast cold starts can dramatically lower your cloud costs — without sacrificing speed or reliability.
Real-world tradeoffs: We talk honestly about Rust’s steeper learning curve and smaller ecosystem, and how to navigate it effectively.
Best practices for shipping production-grade Rust Lambdas: From using Infrastructure as Code to deploying with Cargo Lambda, Luciano walks us through his battle-tested workflows.
Live coding session: Yes — you’ll see me scaffold, test, and deploy a real Lambda function using Rust in minutes.

If you're a backend engineer, cloud architect, or just curious about Rust in the real world, this episode is a must-watch.

Watch the full episode now → Click here

Whether you’re thinking of picking up Rust, exploring serverless, or looking to level up your backend engineering skills, this episode is packed with insights, practical examples, and powerful takeaways.

Let me know what you think after you watch it, and don’t forget to subscribe if you want more conversations like this in your feed.

Did you learn any new things from this newsletter this week? Please reply to this email and let me know. Feedback like this encourages me to keep going.

Remember to start learning backend engineering from our courses:

Get a 60% discount on any of these courses for this easter. Reach out to me (Reply to this mail)

Backend Engineering Resources

Whenever you're ready

There are 4 ways I can help you become a great backend engineer:

2. The MB Academy: The “MB Academy” is a 6-month intensive Advanced Backend Engineering BootCamp to produce great backend engineers.

LAST WORD 👋

How am I doing?

Hit reply and say hello - I'd love to hear from you!

Stay awesome,
Solomon

I moved my newsletter from Substack to Beehiiv, and it's been an amazing journey. Start yours here.

API & API Design: Authentication Methods

Solomom Eseme — Sat, 12 Apr 2025 16:30:00 +0000

Hello “👋

Welcome to another week, another opportunity to become a Great Backend Engineer.

Today’s issue is brought to you by Masteringbackend → A great resource for backend engineers. We offer next-level backend engineering training and exclusive resources.

Before we get started, I have a few announcements:

I have a special gift for you: You will love this one.

The ultimate “Land Your Dream Job” Challenge is here.

We are launching the ultimate guide to land your dream job in any programming language you choose. We are starting with the Python Programming language.

Land your dream Python Job in 90 days by shipping 30 Python projects in 30 days by completing our daily tasks.

It’s a cohort-based and project-focused challenge where you will be challenged to build 30 Python projects in 30 days.

Here is what you will get:

Ship 30+ Python backend projects in 30 days.
Instant Access to all 30+ videos
Access to data structure and algorithm interview kits.
Access our Complete Backend Job Preparation kits (Resume, Cover letter reviews, mock interviews, and job placements).
Join & learn from a thriving community of helpful students & alumni from top companies.

Limited Access. The first 500 students will be at $54, others at $100 (We have only 220 slots left.)

Click here to join the challenge.

Start learning AI in 2025

Everyone talks about AI, but no one has the time to learn it. So, we found the easiest way to learn AI in as little time as possible: The Rundown AI.

It's a free AI newsletter that keeps you up-to-date on the latest AI news, and teaches you how to apply it in just 5 minutes a day.

Plus, complete the quiz after signing up and they’ll recommend the best AI tools, guides, and courses – tailored to your needs.

In our previous issue on Error handling in API Design, we explored best practices for API error handling, structured error responses, and implemented a robust error-handling system in a RESTful API using Node.js and Express.

This issue explores the API design's most widely adopted authentication methods, detailing how they work, when to use them, and how to implement them securely.

Introduction

Application Programming Interfaces (APIs) are fundamental components of modern software systems, enabling applications to communicate, share data, and perform operations programmatically.

As APIs expose system functionality to external clients, ensuring secure access becomes imperative. One of the foundational elements of API security is authentication, the process by which an API verifies the identity of the client making a request.

Authentication ensures that the right users or applications have access to the correct resources. Several authentication methods exist, each designed to serve particular use cases, security levels, and architectural requirements.

Basic Authentication
API Key Authentication
Token-Based Authentication
JWT (JSON Web Tokens)
OAuth 2.0 Authentication
Session-based Authentication

Above are a few authentication methods that we will explore in great detail in upcoming newsletter episodes.

However, we will briefly examine each authentication method to gain a general understanding of the various methods available for API design.

Basic Authentication

Basic Authentication is one of the earliest and simplest authentication mechanisms. In this approach, the client sends the username and password encoded in Base64 as part of the HTTP Authorization header with every request. Although simple to implement, Basic Authentication has significant security limitations.

Basic Authentication

Since the credentials are sent with every request, this method is highly susceptible to interception and should never be used without HTTPS. Basic Authentication is best suited for internal or low-risk APIs where more robust security measures may not be necessary.

app.use((req, res, next) => {
  const authHeader = req.headers.authorization;
  if (!authHeader) return res.status(401).send('Missing Authorization header');

  const base64Credentials = authHeader.split(' ')[1];
  const credentials = Buffer.from(base64Credentials, 'base64').toString('ascii');
  const [username, password] = credentials.split(':');

  if (username === 'admin' && password === 'secret') {
    next();
  } else {
    res.status(401).send('Unauthorized');
  }
});

This code snippet demonstrates Express.js middleware that checks if the username and password are correct for incoming secured requests. The middleware will be added to all secured endpoints to make sure the correct user can access the endpoint.

API Key Authentication

API Key Authentication involves issuing a unique identifier to each client. This key is included in each request, typically as a query parameter or in a custom header, such as x-api-key. The server checks the key against a database or configuration file and grants access if the key is valid.

Token-Based Authentication

While easy to implement and widely supported, API keys do not provide information about the client's identity beyond the key itself. They are static, do not expire automatically, and can be easily leaked if not carefully protected. Nevertheless, they are suitable for server-to-server communication and third-party integrations where simplicity is preferred.

app.use((req, res, next) => {
  const apiKey = req.headers['x-api-key'];
  if (apiKey !== process.env.MY_API_KEY) {
    return res.status(403).send('Forbidden');
  }
  next();
});

This code snippet uses Express middleware to check if the request header contains an API key and if the code is equal to the one saved on the server. If true, give access if not, return an error.

Token-Based Authentication

Token-based authentication improves upon API key authentication by issuing a token to the client after a successful login. This token, usually included in the Authorization header as a Bearer token, is then used to authenticate future requests.

Token-Based Authentication

Unlike API keys, tokens are typically short-lived and can contain encoded metadata. They do not require the server to store session data, making them ideal for stateless architectures. Token-based authentication is a common pattern in modern RESTful APIs.

app.use((req, res, next) => {
  const token = req.headers['authorization']?.split(' ')[1];
  if (token !== 'your-token-here') {
    return res.status(401).send('Invalid token');
  }
  next();
});

The code snippet retrieves your token from your request header and checks to make sure it exists. You can do more checks with the token, for example, making sure itis generated by a real user in your application before giving access to the secured resources.

JWT (JSON Web Tokens)

JWTs, or JSON Web Tokens, are a specific implementation of token-based authentication that package user identity and claims within the token itself. A JWT consists of three parts: the header, the payload, and the signature.

The payload contains claims such as user ID, role, and expiration time. The signature is used to verify the integrity of the token.

JWTs are compact, URL-safe, and can be verified without querying a database, which makes them suitable for scalable, distributed systems. Because they are stateless, JWTs are commonly used in microservices and single-page applications.

const jwt = require('jsonwebtoken');

app.use((req, res, next) => {
  const token = req.headers['authorization']?.split(' ')[1];
  if (!token) return res.status(401).send('Missing token');

  try {
    const decoded = jwt.verify(token, process.env.JWT_SECRET);
    req.user = decoded;
    next();
  } catch (err) {
    res.status(403).send('Invalid or expired token');
  }
});

The code snippet above uses the JWT package to verify a token and retrieves the details of the user who created the token. If the token can not be verified, then access is denied.

OAuth 2.0 Authentication

OAuth 2.0 is a comprehensive framework for delegated authorization. It allows users to authorize third-party applications to access their data without sharing their credentials. OAuth 2.0 defines several flows, including Authorization Code, Implicit, Client Credentials, and Resource Owner Password Credentials, each suited to specific use cases.

In the OAuth 2.0 architecture, the key roles are the Resource Owner (user), the Client (application), the Authorization Server, and the Resource Server. The client obtains an access token from the Authorization Server and uses it to access resources on the Resource Server.

OAuth is ideal for scenarios involving third-party applications, such as "Sign in with Google," where security and delegation are paramount. Implementing OAuth can be complex and typically requires the use of libraries and secure token storage mechanisms.

Session-Based Authentication

While not common in purely RESTful APIs, session-based authentication remains relevant for traditional web applications. Upon successful login, the server creates a session and returns a session ID to the client, usually stored in a browser cookie. Subsequent requests include the cookie, allowing the server to associate the request with a session.

Session-based authentication provides a straightforward user experience and integrates well with web frameworks. However, it requires server-side storage of session state, which can hinder horizontal scalability unless a shared store like Redis is used.

Comparing Authentication Methods

The following diagram illustrates the relationship between different authentication methods, highlighting their security level and use case complexity.

Comparing Authentication Methods

Each authentication method offers a different balance between simplicity, security, and scalability. Basic Authentication and API keys are easier to implement but offer lower security. JWT and OAuth 2.0, while more complex, provide advanced features suitable for modern application architectures.

Conclusion

Selecting the right authentication method is essential in building secure and user-friendly APIs. As systems grow in complexity and user expectations rise, you must balance ease of use, performance, and security. Token-based authentication and JWTs provide scalable, stateless solutions, while OAuth 2.0 addresses advanced authorization requirements.

Authentication is just one layer of API security. When combined with authorization, encryption, logging, and rate limiting, it contributes to a resilient and secure API ecosystem. As you build and scale your APIs, invest in choosing and implementing authentication strategies that align with your security goals and application needs.

Did you learn any new things from this newsletter this week? Please reply to this email and let me know. Feedback like this encourages me to keep going.

Remember to start learning backend engineering from our courses:

Get a 50% discount on any of these courses. Reach out to me (Reply to this mail)

Backend Engineering Resources

Whenever you're ready

There are 4 ways I can help you become a great backend engineer:

2. The MB Academy: The “MB Academy” is a 6-month intensive Advanced Backend Engineering BootCamp to produce great backend engineers.

LAST WORD 👋

How am I doing?

Hit reply and say hello - I'd love to hear from you!

Stay awesome,
Solomon

I moved my newsletter from Substack to Beehiiv, and it's been an amazing journey. Start yours here.

Error Handling in API Design

Solomom Eseme — Sat, 05 Apr 2025 16:00:00 +0000

Hello “👋

Welcome to another week, another opportunity to become a Great Backend Engineer.

Today’s issue is brought to you by Masteringbackend → A great resource for backend engineers. We offer next-level backend engineering training and exclusive resources.

Before we get started, I have a few announcements:

I have a special gift for you: You will love this one.

The ultimate “Land Your Dream Job” Challenge is here.

We are launching the ultimate guide to land your dream job in any programming language you choose. We are starting with the Python Programming language.

Land your dream Python Job in 90 days by shipping 30 Python projects in 30 days by completing our daily tasks.

It’s a cohort-based and project-focused challenge where you will be challenged to build 30 Python projects in 30 days.

Here is what you will get:

Ship 30+ Python backend projects in 30 days.
Instant Access to all 30+ videos
Access to data structure and algorithm interview kits.
Access our Complete Backend Job Preparation kits (Resume, Cover letter reviews, mock interviews, and job placements).
Join & learn from a thriving community of helpful students & alumni from top companies.

Limited Access. The first 500 students will be at $54, others at $100 (We have only 220 slots left.)

Click here to reserve your spot.

Start learning AI in 2025

Everyone talks about AI, but no one has the time to learn it. So, we found the easiest way to learn AI in as little time as possible: The Rundown AI.

It's a free AI newsletter that keeps you up-to-date on the latest AI news, and teaches you how to apply it in just 5 minutes a day.

Plus, complete the quiz after signing up and they’ll recommend the best AI tools, guides, and courses – tailored to your needs.

In our previous issue on HATEOAS in API Design, we explored HATEOAS as a key concept in designing RESTful APIs (Application Programming Interfaces), which implies that the API delivers data and information about available interactions.

In this issue, we'll explore best practices for API error handling, structure error responses, and implement a robust error-handling system in a RESTful API using Node.js and Express.

Introduction

Error handling is a critical aspect of API design that directly impacts an API's usability, reliability, and stability. When APIs fail to handle errors properly, consumers experience unexpected failures, making debugging and maintenance more challenging.

A well-structured error-handling mechanism ensures that issues are properly captured, categorized, and communicated to users, allowing them to respond effectively.

Principles of Effective Error Handling

Use Standard HTTP Status Codes

HTTP status codes provide a universal language for conveying success or failure responses. Some key categories include:

2xx (Success): Indicates a successful request (e.g., 200 OK, 201 Created).
4xx (Client Errors): Indicates issues with the client request (e.g., 400 Bad Request, 401 Unauthorized, 404 Not Found).
5xx (Server Errors): Indicates issues with the API server (e.g., 500 Internal Server Error, 503 Service Unavailable).

Provide Meaningful Error Messages

Error messages should be clear, concise, and actionable. Avoid exposing sensitive internal information; provide details that help clients resolve the issue.

Bad Example:

{
  "error": "Something went wrong."
}

Good Example:

{
  "error": {
    "code": "INVALID_REQUEST",
    "message": "The 'email' field is required.",
    "details": [
      {
        "field": "email",
        "issue": "Missing required field."
      }
    ]
  }
}

Maintain a Consistent Error Response Format

APIs should return errors in a consistent structure to help developers anticipate and handle them easily.

A standard error response format might include:

{
  "error": {
    "code": "RESOURCE_NOT_FOUND",
    "message": "The requested resource was not found.",
    "details": []
  }
}

Log Errors for Debugging

Logging errors helps diagnose issues and improve API reliability. To aid troubleshooting, logs should include error details, timestamps, and request context.

Avoid Overexposing Internal Errors

APIs should not reveal sensitive information like stack traces, database errors, or internal implementation details to clients. Instead, they should provide generic error messages and log detailed errors internally.

Implementing Error Handling in Node.js (Express)

Let's implement a structured error-handling system in a Node.js Express API.

Define a Custom Error Class

class ApiError extends Error {
  constructor(statusCode, message, details = []) {
    super(message);
    this.statusCode = statusCode;
    this.details = details;
  }
}

Create a Centralized Error Handling Middleware

const errorHandler = (err, req, res, next) => {
  const statusCode = err.statusCode || 500;
  const response = {
    error: {
      code: err.code || "INTERNAL_SERVER_ERROR",
      message: err.message || "An unexpected error occurred.",
      details: err.details || []
    }
  };
  
  console.error("API Error:", response); // Log error details
  res.status(statusCode).json(response);
};

module.exports = errorHandler;

Use the Middleware in an Express App

const express = require("express");
const app = express();
const errorHandler = require("./middlewares/errorHandler");

app.use(express.json());

// Example Route with Error Handling
app.get("/user/:id", async (req, res, next) => {
  try {
    const userId = req.params.id;
    if (!userId) {
      throw new ApiError(400, "User ID is required.", [{ field: "id", issue: "Missing parameter" }]);
    }
    // Simulate fetching user (Assume user doesn't exist)
    throw new ApiError(404, "User not found.");
  } catch (error) {
    next(error);
  }
});

// Apply the global error handler
app.use(errorHandler);

app.listen(3000, () => {
  console.log("Server running on port 3000");
});

Handling Different Error Scenarios

Now, let’s explore some types of errors and a proper way to send the response to your client.

Validation Errors

If the request payload is invalid, return 400 Bad Request with details.

{
  "error": {
    "code": "INVALID_REQUEST",
    "message": "Username is required.",
    "details": [{ "field": "username", "issue": "Missing parameter" }]
  }
}

Authentication & Authorization Errors

Unauthorized access attempts should return 401 Unauthorized or 403 Forbidden.

{
  "error": {
    "code": "UNAUTHORIZED",
    "message": "Invalid API key. Access denied."
  }
}

Resource Not Found

When a requested resource doesn’t exist, return 404 Not Found.

{
  "error": {
    "code": "RESOURCE_NOT_FOUND",
    "message": "The requested user was not found."
  }
}

Internal Server Errors

For unexpected server-side issues, return 500 Internal Server Error with a generic message.

{
  "error": {
    "code": "INTERNAL_SERVER_ERROR",
    "message": "An unexpected error occurred. Please try again later."
  }
}

A well-defined error-handling strategy is essential for building robust and user-friendly APIs. By using standard HTTP status codes, providing meaningful error messages, maintaining a consistent response structure, and implementing centralized error handling in Node.js, you can significantly enhance the API's reliability and developer experience.

Following these best practices ensures that your API remains resilient, predictable, and easy to debug, ultimately improving the overall stability of your system.

Did you learn any new things from this newsletter this week? Please reply to this email and let me know. Feedback like this encourages me to keep going.

Remember to start learning backend engineering from our courses:

Get a 50% discount on any of these courses. Reach out to me (Reply to this mail)

Backend Engineering Resources

Whenever you're ready

There are 4 ways I can help you become a great backend engineer:

2. The MB Academy: The “MB Academy” is a 6-month intensive Advanced Backend Engineering BootCamp to produce great backend engineers.

LAST WORD 👋

How am I doing?

Hit reply and say hello - I'd love to hear from you!

Stay awesome,
Solomon

I moved my newsletter from Substack to Beehiiv, and it's been an amazing journey. Start yours here.

Idempotency in API Design

Solomom Eseme — Wed, 02 Apr 2025 17:57:10 +0000

Hello “👋

Welcome to another week, another opportunity to become a Great Backend Engineer.

Today’s issue is brought to you by Masteringbackend → A great resource for backend engineers. We offer next-level backend engineering training and exclusive resources.

Before we get started, I have a few announcements:

I have a special gift for you: You will love this one.

The ultimate “Land Your Dream Job” Challenge is here.

We are launching the ultimate guide to land your dream job in any programming language you choose. We are starting with the Python Programming language.

Land your dream Python Job in 90 days by shipping 30 Python projects in 30 days by completing our daily tasks.

It’s a cohort-based and project-focused challenge where you will be challenged to build 30 Python projects in 30 days.

Here are what you will get:

Ship 30+ Python backend projects in 30 days.
Instant Access to all 30+ videos
Access to data structure and algorithm interview kits.
Access our Complete Backend Job Preparation kits (Resume, Cover letter reviews, mock interviews, and job placements).
Join & learn from a thriving community of helpful students & alumni from top companies.

Limited Access. The first 500 students will be at $54, others at $100 (We have only 320 slots left.)

Click here to reserve your spot.

Looking for unbiased, fact-based news? Join 1440 today.

Subscribe to 1440 today.

Introduction

In API design, idempotency is a crucial concept that ensures multiple identical requests have the same effect as a single request. This means that regardless of how many times a client sends the same request, the server's state remains unchanged beyond the first successful execution.

Idempotency is particularly important for handling retries in distributed systems, network failures, and duplicate requests. It helps improve API fault tolerance, reliability, and consistency, making them more predictable and user-friendly.

Why Is Idempotency Important?

Prevents Duplicate Transactions: Imagine a user initiating a payment request but losing their internet connection mid-process. If the client retries the request, a non-idempotent API could trigger multiple charges. Implementing idempotency ensures that only one transaction is processed.
Supports Safe Retries: Network failures, server crashes, and other transient errors are common in distributed systems. Idempotency allows API clients to retry failed operations without worrying about unintended side effects.
Enhances API Predictability: Developers working with idempotent APIs can confidently send multiple requests without concerns about inconsistencies in the application state.

HTTP Methods and Idempotency

Idempotent Methods:

GET: Fetches a resource without modifying it.
PUT: Updates or creates a resource while ensuring the state remains consistent across multiple requests.
DELETE: Removes a resource but does not change the state beyond the first request.

Non-Idempotent Methods:

POST is often used to create new resources, but repeated requests can lead to duplicate entries unless explicitly handled with idempotency keys.
PATCH: Partially modifies a resource and can cause inconsistencies if not designed carefully.

Implementing Idempotency in APIs

Using Idempotency Keys

An idempotency key is a unique identifier sent with a request to ensure that repeated calls do not lead to unintended side effects. The server stores processed requests and their responses, allowing it to return the same response for subsequent identical requests.

Generate an Idempotency Key: Clients should generate a unique idempotency key (e.g., UUID) and send it in the request headers.
Store Request Results: On the server side, store the idempotency key along with the processed request’s response.
Reuse Previous Responses: If a request with the same idempotency key is received again, return the stored response instead of reprocessing the request.

Example Implementation in Node.js with Redis

const express = require("express");
const redis = require("redis");
const { v4: uuidv4 } = require("uuid");

const app = express();
app.use(express.json());

const client = redis.createClient();

app.post("/process-payment", async (req, res) => {
    const idempotencyKey = req.headers["idempotency-key"];
    if (!idempotencyKey) {
        return res.status(400).json({ error: "Idempotency key is required" });
    }

    client.get(idempotencyKey, (err, data) => {
        if (data) {
            return res.json(JSON.parse(data)); // Return stored response
        }

        // Simulate payment processing
        const response = { success: true, transactionId: uuidv4() };
        client.setex(idempotencyKey, 3600, JSON.stringify(response)); // Store response for 1 hour
        res.json(response);
    });
});

app.listen(3000, () => console.log("Server running on port 3000"));

How It Works:

Clients send an idempotency-key in the request headers.
The server checks Redis for an existing response.
If found, it returns the stored response without reprocessing the request.
If not found, it processes the request, stores the response in Redis, and returns it to the client.

Best Practices for Idempotency

Require Idempotency Keys for Critical Endpoints: Enforce the use of idempotency keys for operations such as payments, user registrations, and order creations.
Set an Expiration for Stored Idempotency Keys: To avoid excessive storage consumption, use TTL (Time-To-Live) on stored idempotency keys.
Ensure Database-Level Constraints: Use unique constraints (e.g., transaction IDs) to prevent duplicate records in case of idempotency failures.
Implement Logging and Monitoring: Track idempotency key usage and failed attempts for debugging and optimizing API performance.

Idempotency is a fundamental concept in robust API design, ensuring safe retries, consistency, and fault tolerance. By using idempotency keys, deduplicating requests, and implementing proper storage mechanisms, APIs can handle repeated requests efficiently and predictably.

By adopting idempotent API practices, you enhance the reliability, security, and usability of your APIs, creating a better experience for both developers and end-users.

Did you learn any new things from this newsletter this week? Please reply to this email and let me know. Feedback like this encourages me to keep going.

Remember to start learning backend engineering from our courses:

Get a 50% discount on any of these courses. Reach out to me (Reply to this mail)

Backend Engineering Resources

Whenever you're ready

There are 4 ways I can help you become a great backend engineer:

2. The MB Academy: The “MB Academy” is a 6-month intensive Advanced Backend Engineering BootCamp to produce great backend engineers.

LAST WORD 👋

How am I doing?

Hit reply and say hello - I'd love to hear from you!

Stay awesome,
Solomon

I moved my newsletter from Substack to Beehiiv, and it's been an amazing journey. Start yours here.

Rate Limiting in API Design

Solomom Eseme — Sat, 29 Mar 2025 19:16:26 +0000

Hello “👋

Welcome to another week, another opportunity to become a Great Backend Engineer.

Today’s issue is brought to you by Masteringbackend → A great resource for backend engineers. We offer next-level backend engineering training and exclusive resources.

Before we get started, I have a few announcements:

I have a special gift for you: You will love this one.

The ultimate “Land Your Dream Job” Challenge is here.

We are launching the ultimate guide to land your dream job in any programming language you choose. We are starting with the Python Programming language.

Land your dream Python Job in 90 days by shipping 30 Python projects in 30 days by completing our daily tasks.

It’s a cohort-based and project-focused challenge where you will be challenged to build 30 Python projects in 30 days.

Here are what you will get:

Ship 30+ Python backend projects in 30 days.
Instant Access to all 30+ videos
Access to data structure and algorithm interview kits.
Access our Complete Backend Job Preparation kits (Resume, Cover letter reviews, mock interviews, and job placements).
Join & learn from a thriving community of helpful students & alumni from top companies.

Limited Access. The first 500 students will be at $54, others at $100 (We have only 320 slots left.)

Click here to reserve your spot.

In our previous issue on Pagination in API Design, we explored the role of pagination in API design, compared different pagination strategies, and provided detailed implementation examples.

In this issue, we’ll explore the importance of rate limiting, common strategies used in API design, and implementation examples in Node.js using Express and Redis.

As APIs become the backbone of modern applications, it is crucial to ensure their stability and prevent abuse. One of the most effective ways to achieve this is rate limiting, a strategy that controls the number of API requests a client can make within a given timeframe.

Rate limiting is key in managing resource allocation, preventing system overload, and ensuring fair access to resources for all consumers.

Why Rate Limiting Matters in API Design

Without rate limiting, APIs are vulnerable to excessive requests, which can lead to:

System Overload – High traffic can exhaust server resources, causing slow responses or outages.
Denial-of-Service (DoS) Attacks: Malicious users or bots can flood the API with requests, disrupting service for legitimate users.
Fair Usage Enforcement – Prevents a single client from consuming disproportionate resources.
Cost Control – APIs with pay-per-use models must enforce limits to avoid excessive cloud costs.

By enforcing rate limits, APIs maintain performance, security, and equitable access for all users.

Common Rate Limiting Strategies

Rate limiting is typically implemented using one of the following techniques:

Fixed Window Rate Limiting

This is a simple approach in which requests are counted within a fixed time window (e.g., 100 requests per minute). Once the limit is exceeded, further requests are blocked until the next time window starts.

Example: A user can make 100 requests per minute. If they exceed this limit at 30 seconds, they must wait until the next minute to make more requests.

This strategy is good for Simplicity and predictable enforcement.

Sliding Window Rate Limiting

Instead of resetting limits at fixed intervals, this approach considers a rolling timeframe. More accurate and smooth enforcement compared to fixed window limiting.

Example: If a limit of 100 requests per minute is set, requests are checked within the last 60 seconds at any given moment.

This strategy is good for APIs with dynamic user activity.

Token Bucket Algorithm

Each client is given a bucket with tokens representing API requests. Every request consumes a token; tokens are replenished at a fixed rate. If the bucket is empty, requests are denied until more tokens are added.

Example: A client gets 10 tokens per second. If they make 50 requests in one second, they must wait for tokens to replenish.

This rate-limiting strategy is good for APIs that need a burst tolerance while enforcing limits.

Leaky Bucket Algorithm

It is similar to the token bucket but enforces a steady request rate. Requests are processed at a fixed rate, and excess requests are queued. This strategy is good for APIs requiring smooth request flow without sudden bursts.

Rate Limiting by User or IP Address

Limits requests based on user authentication tokens or IP addresses. Ensures that individual clients do not abuse the API. This strategy is good for Multi-user environments where fairness is important.

Implementing Rate Limiting in Node.js with Express & Redis

One of the best ways to implement rate limiting efficiently is by using Redis, a fast in-memory datastore, alongside Express.js.

Step 1: Install Dependencies

npm install express rate-limit ioredis express-rate-limit-redis

Step 2: Set Up Express and Redis

const express = require('express');
const rateLimit = require('express-rate-limit');
const RedisStore = require('rate-limit-redis');
const { createClient } = require('ioredis');

const app = express();
const redisClient = createClient({ host: 'localhost', port: 6379 });

const limiter = rateLimit({
    store: new RedisStore({
        sendCommand: (...args) => redisClient.call(...args),
    }),
    windowMs: 1 * 60 * 1000, // 1 minute
    max: 100, // Limit each IP to 100 requests per minute
    message: 'Too many requests, please try again later.',
});

app.use('/api', limiter);

app.get('/api/data', (req, res) => {
    res.json({ message: 'Success! You are within the rate limit.' });
});

app.listen(3000, () => console.log('Server running on port 3000'));

How This Works

A rate limit of 100 requests per minute is enforced.
If a user exceeds this limit, they receive a 429 Too Many Requests response.
Redis is used to store request counts for each client, ensuring efficiency and scalability.

Best Practices for Rate Limiting

Use Adaptive Rate Limits – Adjust limits based on user behavior (e.g., stricter limits for anonymous users, relaxed limits for authenticated users).
Implement API Key-Based Limits – Assign different limits based on API usage tiers (free vs. premium users).
Use Headers to Inform Clients – Return headers like X-RateLimit-Limit, X-RateLimit-Remaining, and X-RateLimit-Reset to help clients manage requests.
Combine Rate Limiting with Other Security Measures – Use it alongside authentication, logging, and monitoring to enhance security.
Monitor and Analyze Requests – Store rate limit logs to detect patterns of abuse and adjust policies dynamically.

Conclusion

Rate limiting is essential to API design, ensuring fairness, security, and system stability. By implementing fixed window, sliding window, token bucket, or leaky bucket strategies, APIs can effectively manage load and prevent abuse.

Using tools like Express and Redis, backend engineers can enforce rate limits efficiently while providing a smooth experience for legitimate users. Thoughtful design and best practices help maintain high availability and performance.

Did you learn any new things from this newsletter this week? Please reply to this email and let me know. Feedback like this encourages me to keep going.

Remember to start learning backend engineering from our courses:

Get a 50% discount on any of these courses. Reach out to me (Reply to this mail)

Backend Engineering Resources

Whenever you're ready

There are 4 ways I can help you become a great backend engineer:

2. The MB Academy: The “MB Academy” is a 6-month intensive Advanced Backend Engineering BootCamp to produce great backend engineers.

LAST WORD 👋

How am I doing?

Hit reply and say hello - I'd love to hear from you!

Stay awesome,
Solomon

I moved my newsletter from Substack to Beehiiv, and it's been an amazing journey. Start yours here.

Pagination in API Design

Solomom Eseme — Tue, 18 Feb 2025 17:47:59 +0000

Hello “👋

Welcome to another week, another opportunity to become a Great Backend Engineer.

Today’s issue is brought to you by Masteringbackend → A great resource for backend engineers. We offer next-level backend engineering training and exclusive resources.

Before we get started, I have a few announcements:

I have a special gift for you: You will love this one.

The ultimate “Land Your Dream Job” Challenge is here.

We are launching the ultimate guide to land your dream job in any programming language you choose. We are starting with the Python Programming language.

Land your dream Python Job in 90 days by shipping 30 Python projects in 30 days by completing our daily tasks.

It’s a cohort-based and project-focused challenge where you will be challenged to build 30 Python projects in 30 days.

Here are what you will get:

Ship 30+ Python backend projects in 30 days.
Instant Access to all 30+ videos
Access to data structure and algorithm interview kits.
Access our Complete Backend Job Preparation kits (Resume, Cover letter reviews, mock interviews, and job placements).
Join & learn from a thriving community of helpful students & alumni from top companies.

Limited Access. First 400 students at $54 others at $100 (We have only 120 slots left.)

Click here to reserve your spot.

There’s a reason 400,000 professionals read this daily.

Join The AI Report, trusted by 400,000+ professionals at Google, Microsoft, and OpenAI. Get daily insights, tools, and strategies to master practical AI skills that drive results.

In our previous issue on Building JSON/RESTful APIs, we explored the principles of designing scalable, maintainable, and efficient APIs using REST and JSON.

However, as your API scales and begins handling large datasets, it becomes impractical to return all data in a single response.

This is where pagination comes into play.

In this issue, we will explore the role of pagination in API design, compare different pagination strategies, and provide detailed implementation examples.

By the end, you'll have a solid understanding of how to implement pagination effectively in your JSON/RESTful APIs.

Why Pagination Matters in API Design

APIs often need to return large datasets, such as:

A list of users in a system
Products in an e-commerce catalog
Log entries in a monitoring system

If an API were to return thousands (or millions) of records in a single response, the consequences would be severe:

Slow Performance – Large responses increase response time and slow down both the server and the client.
High Memory Consumption – Loading all records into memory at once puts unnecessary strain on server resources.
Network Overload – Transferring huge JSON responses can cause bandwidth congestion and increase API latency.

Pagination solves these problems by breaking the dataset into smaller chunks (pages), allowing clients to request only what they need.

Pagination Strategies

There are several ways to implement pagination, each with its trade-offs:

Limit-Offset Pagination (Basic Pagination)
Cursor-Based Pagination
Time-Based Pagination
Keyset Pagination

Let's break them down in detail.

Limit-Offset Pagination (Traditional Approach)

How it Works:

Clients specify two query parameters:
- limit (number of records per page)
- offset (starting position of the data)
The server retrieves records starting from the given offset up to the limit.

Example API Call:

GET /users?limit=10&offset=20

SQL Query Example:

SELECT * FROM users ORDER BY id LIMIT 10 OFFSET 20;

Implementation in Node.js (Express + Prisma ORM)

app.get('/users', async (req, res) => {
    const limit = parseInt(req.query.limit) || 10;
    const offset = parseInt(req.query.offset) || 0;

    const users = await prisma.user.findMany({
        skip: offset,
        take: limit,
        orderBy: { id: 'asc' },
    });

    res.json({ users, limit, offset });
});

Pros:

Simple to implement and understand
Easy to integrate with SQL databases

Cons:

Performance issues with large datasets (OFFSET grows linearly)
Skipping rows is inefficient (especially with big tables)

Cursor-Based Pagination

This strategy is great for large datasets and a very efficient pagination strategy for millions and billions of datasets.

How it Works:

Instead of specifying an offset, the client provides a cursor (a unique identifier for the last seen record).
The API retrieves the next N records starting after the cursor.

Example API Call:

GET /users?limit=10&cursor=100

SQL Query Example:

SELECT * FROM users WHERE id > 100 ORDER BY id ASC LIMIT 10;

This query will retrieve all the items where the ID is greater than 100 and will stop at the 10th element because of the LIMIT keyword. Then next iteration will start from 111 like that till it finishes the list.

Implementation in Node.js (Express + Prisma)

app.get('/users', async (req, res) => {
    const limit = parseInt(req.query.limit) || 10;
    const cursor = req.query.cursor ? parseInt(req.query.cursor) : null;

    const users = await prisma.user.findMany({
        where: cursor ? { id: { gt: cursor } } : {},
        take: limit,
        orderBy: { id: 'asc' },
    });

    // Set next cursor (last item ID)
    const nextCursor = users.length ? users[users.length - 1].id : null;

    res.json({ users, nextCursor });
});

Pros:

Efficient for large datasets (avoids expensive OFFSET operations)
Fast and scalable

Cons:

Requires a unique and indexed sorting field (like id)
Cannot jump to arbitrary pages easily

Time-Based Pagination

Time-based pagination is mostly used for real-time pagination, and it’s very efficient when used with WebSocket and other real-time applications.

How it Works:

Instead of an offset, pagination is based on timestamps.
Clients fetch records newer or older than a given timestamp.

Example API Call:

GET /logs?limit=10&after=2024-02-17T12:30:00Z

SQL Query Example:

SELECT * FROM logs WHERE created_at > '2024-02-17T12:30:00Z' ORDER BY created_at ASC LIMIT 10;

Implementation in Node.js (Express + Prisma)

app.get('/logs', async (req, res) => {
    const limit = parseInt(req.query.limit) || 10;
    const after = req.query.after ? new Date(req.query.after) : null;

    const logs = await prisma.logs.findMany({
        where: after ? { createdAt: { gt: after } } : {},
        take: limit,
        orderBy: { createdAt: 'asc' },
    });

    const nextAfter = logs.length ? logs[logs.length - 1].createdAt : null;

    res.json({ logs, nextAfter });
});

Pros:

Ideal for real-time updates (chat apps, logs, notifications)
Efficient indexing with timestamp columns

Cons:

Prone to issues with time zone inconsistencies
Does not support jumping to specific pages

Keyset Pagination

Keyset-based pagination is an alternative cursor-based pagination.

How it Works:

Uses multiple unique columns (e.g., id + created_at) to determine the next page.
It prevents inconsistencies when sorting by fields that are not unique.

Example Query:

SELECT * FROM orders 
WHERE (created_at, id) > ('2024-02-17T12:30:00Z', 100) 
ORDER BY created_at ASC, id ASC LIMIT 10;

Pros:

More stable sorting for non-unique fields
Efficient for ordered datasets

Cons:

More complex implementation

Best Practices for API Pagination

Choose the Right Pagination Strategy
- Use limit-offset for small datasets.
- Use cursor-based for large datasets.
- Use time-based for real-time applications.
Return Pagination Metadata
- Always include pagination metadata (nextCursor, nextPage, totalPages).
Optimize Database Queries
- Use indexed fields for filtering (e.g., id, created_at).
- Avoid OFFSET in large tables when possible.
Consider Client Usability
- Ensure clients can navigate easily (next, previous, first, last links).

Pagination is an essential component of API design, ensuring efficient data retrieval while maintaining performance and scalability. Choosing the right pagination strategy depends on your API's requirements—limit-offset is simple but inefficient for large datasets, while cursor-based and keyset pagination is ideal for high-performance applications.

Implementing these best practices, backend engineers can design scalable, performant APIs that provide clients with a seamless experience while optimizing backend resources.

Did you learn any new things from this newsletter this week? Please reply to this email and let me know. Feedback like this encourages me to keep going.

Remember to start learning backend engineering from our courses:

Get a 50% discount on any of these courses. Reach out to me (Reply to this mail)

Top 5 Remote Backend Jobs this week

Here are the top 5 Backend Jobs you can apply to now.

👨‍💻 FaceUp
✍️ Node.js Backend Developer for FaceUp
📍Remote, Prague, Brno
💰 Click on Apply for salary details
Click here to Apply for this role.

👨‍💻 ROIVENUE
✍️ Full-stack Software Developer
📍Remote, Prague
💰 Click on Apply for salary details
Click here to Apply for this role.

👨‍💻 Homepage
✍️ Backend Developer PHP
📍Remote, Worldwide
💰 Click on Apply for salary details
Click here to Apply for this role.

Want more Remote Backend Jobs? Visit GetBackendJobs.com

Backend Engineering Resources

Whenever you're ready

There are 4 ways I can help you become a great backend engineer:

2. The MB Academy: The “MB Academy” is a 6-month intensive Advanced Backend Engineering BootCamp to produce great backend engineers.

LAST WORD 👋

How am I doing?

Hit reply and say hello - I'd love to hear from you!

Stay awesome,
Solomon

I moved my newsletter from Substack to Beehiiv, and it's been an amazing journey. Start yours here.

Building RESTful APIs

Solomom Eseme — Sat, 08 Feb 2025 15:07:32 +0000

Hello “👋

Welcome to another week, another opportunity to become a Great Backend Engineer.

Today’s issue is brought to you by Masteringbackend → A great resource for backend engineers. We offer next-level backend engineering training and exclusive resources.

Before we get started, I have a few announcements:

I have a special gift for you: You will love this one.

The ultimate “Land Your Dream Job” Challenge is here.

We are launching the ultimate guide to land your dream job in any programming language you choose. We are starting with the Python Programming language.

Land your dream Python Job in 90 days by shipping 30 Python projects in 30 days by completing our daily tasks.

It’s a cohort-based and project-focused challenge where you will be challenged to build 30 Python projects in 30 days.

Here are what you will get:

Ship 30+ Python backend projects in 30 days.
Instant Access to all 30+ videos
Access to data structure and algorithm interview kits.
Access our Complete Backend Job Preparation kits (Resume, Cover letter reviews, mock interviews, and job placements).
Join & learn from a thriving community of helpful students & alumni from top companies.

Limited Access. First 400 students at $54 others at $100 (We have only 120 slots left.)

Reply “Challenge” and I will send you a link to reserve your spot.

Check out our sponsor for this episode:

Writer RAG tool: build production-ready RAG apps in minutes

Writer RAG Tool: build production-ready RAG apps in minutes with simple API calls.
Knowledge Graph integration for intelligent data retrieval and AI-powered interactions.
Streamlined full-stack platform eliminates complex setups for scalable, accurate AI workflows.

Learn more about our production ready RAG tooling here.

In the last newsletter issue, we discussed How Server-Sent Events (SSE) Work and explained everything you need to know about them. Most importantly, we explored how Server-Sent Events (SSE) works.

Let’s deep dive into Building a RESTful API today and learn the intricacies of building an API, including Pagination, Rate Limiting, Idempotency, HATEOAS, Error Handling, etc. We will cover these topics in each newsletter issue. But for today, let’s look at how to build a simple RESTful/JSON API using Express.js.

Understanding the Fundamentals

What Is REST?

REST is an architectural style that defines a set of constraints and principles for building distributed systems. At its core, REST emphasizes:

Statelessness: Each request from a client must contain all the necessary information for the server to process it. The server does not store client context between requests, enabling scalability and reliability.
Resource Orientation: Resources (such as users, products, or orders) are identified by unique URLs. Operations on these resources are performed using standard HTTP methods (GET, POST, PUT, DELETE).
Uniform Interface: A consistent interface simplifies the interaction between clients and servers, making APIs easier to use and understand.

Why JSON?

JSON is a lightweight data interchange format known for its simplicity and readability. Its key attributes include:

Ease of Use: JSON's key-value pair structure is straightforward and intuitive, making it accessible to both humans and machines.
Lightweight: Its minimal syntax reduces payload size, leading to faster data transmission.
Language Agnostic: Almost every modern programming language supports JSON, ensuring seamless integration across diverse systems.

By leveraging JSON as the medium for data exchange, RESTful APIs become easier to work with and integrate, helping you to build interconnected systems more efficiently.

Key Principles in Building APIs

When designing and implementing a JSON/RESTful API, consider the following architectural constraints and design principles:

Resource Identification and Naming Conventions

Clear Endpoints: Use meaningful, pluralized nouns to represent collections (e.g., /users, /orders) and singular nouns for individual resources (e.g., /users/123).
Consistent URL Structure: Keep endpoint naming uniform and intuitive to facilitate discovery and ease of use.

Utilizing Standard HTTP Methods

RESTful APIs use standard HTTP methods to perform operations on resources:

GET: Retrieve resource representations.
POST: Create a new resource.
PUT/PATCH: Update an existing resource.
DELETE: Remove a resource.

Using these methods consistently aligns your API with RESTful principles and simplifies client-server interactions.

Embracing Statelessness

Every request must be self-contained:

Complete Information: Each request should carry all the data needed for processing, such as authentication tokens or query parameters.
Scalability: Stateless interactions make it easier to distribute load across multiple servers and handle high traffic volumes.

Data Representation and Format

JSON as the Data Format: JSON’s widespread acceptance makes it ideal for information interchange. It’s human-readable and easily parsed by client-side applications.
Schema and Versioning: Document the structure of your JSON payloads using schemas and maintain versioned endpoints (e.g., /api/v1/users) to manage changes without disrupting existing clients.

Error Handling and Status Codes

HTTP Status Codes: Use standard HTTP status codes to indicate the outcome of a request:
- 200 OK for successful requests.
- 201 Created for successful resource creation.
- 400 Bad Request for malformed requests.
- 404 Not Found when resources cannot be found.
- 500 Internal Server Error for unexpected failures.
Structured Error Messages: Return JSON-formatted error messages that provide clear details about what went wrong.

Practical Example

This example will be built with Node.js because it is simple and easy to understand by everyone. For those, who do not write Node.js, you can use ChatGPT to cover the code to any programming language of your choice.

Below is a simple implementation using Node.js and Express to illustrate how you can build a JSON/RESTful API.

Step 1: Setting Up the Environment

First, create a new Node.js project and install Express:

mkdir json-rest-api
cd json-rest-api
npm init -y
npm install express

Step 2: Create the API Server

Create a file named app.js:

const express = require('express');
const app = express();

// Middleware to parse JSON bodies
app.use(express.json());

// Sample in-memory data store
let users = [
  { id: 1, name: 'Alice', email: 'alice@example.com' },
  { id: 2, name: 'Bob', email: 'bob@example.com' }
];

// GET /users - Retrieve all users
app.get('/users', (req, res) => {
  res.status(200).json(users);
});

// GET /users/:id - Retrieve a user by ID
app.get('/users/:id', (req, res) => {
  const user = users.find(u => u.id === parseInt(req.params.id, 10));
  if (user) {
    res.status(200).json(user);
  } else {
    res.status(404).json({ error: 'User not found' });
  }
});

// POST /users - Create a new user
app.post('/users', (req, res) => {
  const newUser = {
    id: Date.now(),
    name: req.body.name,
    email: req.body.email
  };
  users.push(newUser);
  res.status(201).json(newUser);
});

// PUT /users/:id - Update an existing user
app.put('/users/:id', (req, res) => {
  const userIndex = users.findIndex(u => u.id === parseInt(req.params.id, 10));
  if (userIndex !== -1) {
    users[userIndex] = { ...users[userIndex], ...req.body };
    res.status(200).json(users[userIndex]);
  } else {
    res.status(404).json({ error: 'User not found' });
  }
});

// DELETE /users/:id - Delete a user
app.delete('/users/:id', (req, res) => {
  const userIndex = users.findIndex(u => u.id === parseInt(req.params.id, 10));
  if (userIndex !== -1) {
    const deletedUser = users.splice(userIndex, 1);
    res.status(200).json({ message: 'User deleted', user: deletedUser });
  } else {
    res.status(404).json({ error: 'User not found' });
  }
});

// Start the server
const PORT = process.env.PORT || 3000;
app.listen(PORT, () => console.log(`API running on http://localhost:${PORT}`));

This is a simple example of a demo RESTful where we are using an in-memory array storage mechanism instead of a database. Every other action is about manipulating the array to get results and convert them to JSON.

Benefits of Building JSON/RESTful APIs

Scalability: Stateless interactions allow the API to scale horizontally across multiple servers.
Maintainability: Clear separation of concerns and adherence to REST principles makes the API easier to maintain.
Integration: Standardized protocols and data formats ensure seamless integration with other systems and services.
Performance: Lightweight JSON reduces data overhead, leading to faster communication between client and server.

Building APIs involves more than just sending and receiving data. It’s about adhering to principles that ensure your API is scalable, maintainable, and easily integrated with other systems.

In the upcoming series, we will explore the following concepts Pagination, Rate Limiting, Idempotency, HATEOAS, and Error Handling, and expand the simple API above into a more robust and enterprise-ready API. Whether you're developing a simple application or a complex enterprise system, mastering these principles is essential for creating robust and future-proof APIs.

Did you learn any new things from this newsletter this week? Please reply to this email and let me know. Feedback like this encourages me to keep going.

See you on Next Week.

Remember to start learning backend engineering from our courses:

Get a 50% discount on any of these courses. Reach out to me (Reply to this mail)

Top 5 Remote Backend Jobs this week

Here are the top 5 Backend Jobs you can apply to now.

👨‍💻 FaceUp
✍️ Node.js Backend Developer for FaceUp
📍Remote, Prague, Brno
💰 Click on Apply for salary details
Click here to Apply for this role.

👨‍💻 ROIVENUE
✍️ Full-stack Software Developer
📍Remote, Prague
💰 Click on Apply for salary details
Click here to Apply for this role.

👨‍💻 Homepage
✍️ Backend Developer PHP
📍Remote, Worldwide
💰 Click on Apply for salary details
Click here to Apply for this role.

👨‍💻 Monzo
✍️ Backend Engineer
📍Remote
💰 Click on Apply for salary details
Click here to Apply for this role.

Want more Remote Backend Jobs? Visit GetBackendJobs.com

Backend Engineering Resources

Whenever you're ready

There are 4 ways I can help you become a great backend engineer:

2. The MB Academy: The “MB Academy” is a 6-month intensive Advanced Backend Engineering BootCamp to produce great backend engineers.

LAST WORD 👋

How am I doing?

Hit reply and say hello - I'd love to hear from you!

Stay awesome,
Solomon

I moved my newsletter from Substack to Beehiiv, and it's been an amazing journey. Start yours here.

API and API Design: How Do Server-Sent Events (SSE) Work

Solomom Eseme — Wed, 05 Feb 2025 16:23:03 +0000

Hello “👋

Welcome to another week, another opportunity to become a Great Backend Engineer.

Today’s issue is brought to you by Masteringbackend → A great resource for backend engineers. We offer next-level backend engineering training and exclusive resources.

Before we get started, I have a few announcements:

I have a special gift for you: You will love this one.

The ultimate Land Your Dream Job Challenge is here.

We are launching the ultimate guide to land your dream job in any programming language you choose. We are starting with the Python Programming language.

Land your dream Python Job in 90 days by shipping 30 Python projects in 30 days by completing our daily tasks.

It’s a cohort-based and project-focused challenge where you will be challenged to build 30 Python projects in 30 days.

Here are what you will get:

Ship 30+ Python backend projects in 30 days.
Instant Access to all 30+ videos
Access to data structure and algorithm interview kits.
Access our Complete Backend Job Preparation kits (Resume, Cover letter reviews, mock interviews, and job placements).
Join & learn from a thriving community of helpful students & alumni from top companies.

Limited Access. First 400 students at $54 others at $100 (We have only 94 slots left.)

Reply “Challenge” and I will send you a link to reserve your spot.

Check out our sponsor for this episode:

Start learning AI in 2025

Everyone talks about AI, but no one has the time to learn it. So, we found the easiest way to learn AI in as little time as possible: The Rundown AI.

It's a free AI newsletter that keeps you up-to-date on the latest AI news, and teaches you how to apply it in just 5 minutes a day.

Plus, complete the quiz after signing up and they’ll recommend the best AI tools, guides, and courses – tailored to your needs.

In the last newsletter issue, we discussed How WebSockets Works and explained everything you need to know about them. Most importantly, we identified two important types of Real-time APIs: WebSockets and Server-Sent Events (SSE).

Let’s deep dive into Server-Sent Events (SSE) today and learn the intricacies of SSE, including a simple program that shows how to implement SSE into our projects for real-time application.

Overview of Server-Sent Events (SSE)

Server-sent events (SSE) have emerged as a powerful and efficient mechanism for unidirectional data streaming.

Unlike traditional request-response architectures, where a client continuously polls for updates, SSE enables the server to push data to clients whenever new information becomes available.

This makes SSE particularly valuable for real-time applications such as live news updates, real-time gaming, stock market feeds, and live-streaming services.

This issue will explore how SSE works, its advantages over other real-time communication methods, and best practices for designing SSE-powered APIs.

What Are Server-Sent Events (SSE)?

Server-sent events (SSE) are a real-time API technology built on standard HTTP that allows servers to continuously push updates to clients over a single connection. It uses the EventSource API, which is natively supported by most modern browsers, making it an accessible and efficient solution for real-time communication.

Unlike WebSockets, which support full-duplex communication (bi-directional data exchange), SSE follows a one-way data flow:

The server continuously sends updates to the client.
The client passively listens for updates without needing to request them explicitly.

This push-based architecture reduces network overhead and ensures clients receive real-time updates without repeated polling.

Now we understand what SSE is and how it differs from WebSockets as we discussed in the previous issue. Next, let’s explore how SSE works.

How Does SSE Work?

SSE operates using a long-lived HTTP connection where the server streams events to the client. The interaction follows these key steps:

Client Initiates the Connection: The client opens an HTTP connection to the server using the EventSource API. The server keeps this connection open and sends updates as events occur.
Server Pushes Data: Instead of waiting for a new client request, the server automatically pushes data to all connected clients when an update is available.
Client Receives Events in Real-Time: The client listens for incoming messages and processes them as they arrive. The connection remains open until the server closes it or the client disconnects.

SSE Request and Response Flow

Next, let’s look at the request and response flow of a server-sent event and create a simple implementation using Node.js.

Client-Side: Opening an SSE Connection

Using JavaScript’s EventSource API, a client can easily subscribe to an SSE stream:

const eventSource = new EventSource('http://example.com/stream');

eventSource.onmessage = function(event) {
    console.log('New event received:', event.data);
};

eventSource.onerror = function() {
    console.log('Connection lost, attempting to reconnect...');
};

In the code snippet above, you see that we use the JavaScript’s EventSource API to create a new event and listen to incoming messages from the server using the onmessage event.

Also, we use the onerror event to catch any errors during connection and other errors that might occur during interactions. If the connection is lost, most browsers will automatically retry the connection.

Server-Side: Streaming Events in Node.js

Next, let’s look at the server-side implementation of SSE in Node.js:

const express = require('express');
const app = express();

app.get('/stream', (req, res) => {
    res.setHeader('Content-Type', 'text/event-stream');
    res.setHeader('Cache-Control', 'no-cache');
    res.setHeader('Connection', 'keep-alive');

    let counter = 0;

    const interval = setInterval(() => {
        counter++;
        res.write(`data: {"message": "Update #${counter}", "timestamp": ${Date.now()}}\n\n`);
    }, 3000);

    req.on('close', () => {
        clearInterval(interval);
    });
});

app.listen(3000, () => console.log('SSE server running on http://localhost:3000/stream'));

In the code snippet above, The server sets text/event-stream as the response type to indicate an SSE connection. res.write() is used to continuously push new updates. The connection remains open until the client disconnects or the server closes it.

Use Cases for SSE

SSE is ideal for applications where real-time updates flow in a single direction from server to client:

Live News and Social Media Feeds: Users receive instant updates on breaking news or trending posts.
Stock Market and Cryptocurrency Price Updates: Investors see real-time fluctuations without refreshing their dashboards.
Live Sports Scores: Fans get live score updates as matches progress.
IoT Sensor Data Monitoring: Devices stream sensor data to dashboards in real-time.
Customer Support Notifications: Agents receive instant alerts when new customer inquiries arrive.

SSE vs. WebSockets vs. Polling

Let’s see a clear distinction between SSE, Websockets, and polling.

Feature	SSE (Server-Sent Events)	WebSockets	Polling
Direction	One-way (Server → Client)	Bi-directional	Client repeatedly requests updates
Efficiency	Efficient for server push	Ideal for real-time interaction	High overhead
Use Case	Live updates (news, stocks)	Chat apps, gaming, real-time collaboration	Basic data sync
Transport	Standard HTTP	TCP-based WebSocket protocol	Standard HTTP
Browser Support	Native in most browsers	Requires WebSocket support	Works everywhere
Auto-Reconnect	Yes (handled by browser)	Requires manual handling	Requires manual handling

When to use SSE: When only the server needs to send updates (e.g., live feeds, notifications). When simplicity and efficiency are priorities.
When to use WebSockets: When the client also needs to send data continuously (e.g., chat applications, online gaming).
When to use Polling: When real-time is not critical, but updates are needed periodically (e.g., checking for new emails every 5 minutes).

Best Practices for Implementing SSE

Use Proper Headers: Always set Content-Type: text/event-stream, Cache-Control: no-cache, and Connection: keep-alive to ensure smooth streaming.
Handle Connection Loss Gracefully: Clients should automatically reconnect when the connection is lost. SSE supports built-in reconnection, but you can use eventSource.onerror to handle failures.
Keep Messages Lightweight: Only send necessary data to minimize bandwidth usage. Use JSON serialization for structured messages.
Optimize Server Performance: Limit the number of active connections to prevent excessive resource consumption. Consider load balancing for large-scale applications.
Secure the Connection: Use HTTPS (wss://) for encrypted communication. Implement authentication tokens to control access.

Server-sent events (SSE) provide a powerful, efficient, and lightweight way for servers to push real-time updates to clients. By leveraging persistent connections, SSE reduces the need for unnecessary polling, leading to better performance and scalability.

While it may not be suitable for every use case (especially those requiring bi-directional communication), SSE excels in scenarios where the server continuously streams real-time updates—such as live news, stock market tracking, or IoT monitoring.

Did you learn any new things from this newsletter this week? Please reply to this email and let me know. Feedback like this encourages me to keep going.

See you on Next Week.

Remember to start learning backend engineering from our courses:

Get a 50% discount on any of these courses. Reach out to me (Reply to this mail)

Top 5 Remote Backend Jobs this week

Here are the top 5 Backend Jobs you can apply to now.

👨‍💻 FaceUp
✍️ Node.js Backend Developer for FaceUp
📍Remote, Prague, Brno
💰 Click on Apply for salary details
Click here to Apply for this role.

👨‍💻 ROIVENUE
✍️ Full-stack Software Developer
📍Remote, Prague
💰 Click on Apply for salary details
Click here to Apply for this role.

👨‍💻 Homepage
✍️ Backend Developer PHP
📍Remote, Worldwide
💰 Click on Apply for salary details
Click here to Apply for this role.

👨‍💻 Monzo
✍️ Backend Engineer
📍Remote
💰 Click on Apply for salary details
Click here to Apply for this role.

Want more Remote Backend Jobs? Visit GetBackendJobs.com

Backend Engineering Resources

Whenever you're ready

There are 4 ways I can help you become a great backend engineer:

2. The MB Academy: The “MB Academy” is a 6-month intensive Advanced Backend Engineering BootCamp to produce great backend engineers.

LAST WORD 👋

How am I doing?

Hit reply and say hello - I'd love to hear from you!

Stay awesome,
Solomon

I moved my newsletter from Substack to Beehiiv, and it's been an amazing journey. Start yours here.

API and API Design: How WebSockets Works

Solomom Eseme — Thu, 23 Jan 2025 11:22:23 +0000

Hello “👋

Welcome to another week, another opportunity to become a Great Backend Engineer.

Today’s issue is brought to you by Masteringbackend → A great resource for backend engineers. We offer next-level backend engineering training and exclusive resources.

Before we get started, I have a few announcements:

I have a special gift for you: You will love this one.

The ultimate Land Your Dream Job Challenge is here.

We are launching the ultimate guide to land your dream job in any programming language you choose. We are starting with the Python Programming language.

Land your dream Python Job in 90 days by shipping 30 Python projects in 30 days by completing our daily tasks.

It’s a cohort-based and project-focused challenge where you will be challenged to build 30 Python projects in 30 days.

Here are what you will get:

Ship 30+ Python backend projects in 30 days.
Instant Access to all 30+ videos
Access to data structure and algorithm interview kits.
Access our Complete Backend Job Preparation kits (Resume, Cover letter reviews, mock interviews, and job placements).
Join & learn from a thriving community of helpful students & alumni from top companies.

Limited Access. First 400 students at $54 others at $100 (We have only 113 slots left.)

Reply “Challenge” and I will send you a link to reserve your spot.

Check out our sponsor for this episode:

Start learning AI in 2025

Everyone talks about AI, but no one has the time to learn it. So, we found the easiest way to learn AI in as little time as possible: The Rundown AI.

It's a free AI newsletter that keeps you up-to-date on the latest AI news, and teaches you how to apply it in just 5 minutes a day.

Plus, complete the quiz after signing up and they’ll recommend the best AI tools, guides, and courses – tailored to your needs.

In the last newsletter issue, we discussed Real-time APIs and explained everything you need to know about them. Most importantly, we identified two important types of Real-time APIs: WebSockets and Server-Sent Events (SSE).

Let’s deep dive into WebSockets today and learn the intricacies of Websockets, including a simple program that shows how to implement Websockets into our projects for real-time application.

Overview of Websockets

Web Sockets provide a long-lived connection between a client and a server over which messages can be transmitted bi-directionally in real time.

Websockets are very popular because messages can be sent bidirectionally, which means both the server and the client can send and receive messages in real-time.

WebSockets play a pivotal role in building real-time APIs by providing a faster and more efficient communication channel than traditional HTTP.

They are widely adopted in scenarios that demand instantaneous data transfer—such as chat applications, live sports updates, and real-time analytics—because they enable seamless, bidirectional communication.

This shift from conventional HTTP-based API design to a WebSocket-driven approach creates more responsive, dynamic, and efficient APIs capable of handling real-time data.

What Are WebSockets?

WebSockets are a communication protocol designed to support full-duplex (two-way) communication between client and server over a single, long-lived connection.

Introduced as part of the HTML5 specification, they address the limitations of traditional HTTP by allowing servers to push data to clients without waiting for a client request.

A simple design right? but very powerful when you think of the numerous complex real-time applications that have been built with it.

Let’s take a look at some of the key advantages of Websockets:

Key Advantages of WebSockets

Real-time, Two-Way Communication: Data can travel in both directions (client-to-server and server-to-client) simultaneously, making it ideal for real-time applications.
Reduced Latency: Once the connection is established, messages flow with minimal overhead, leading to faster data exchange.
Efficient Use of Resources: WebSockets eliminates the need for repetitive HTTP handshakes and polling, lowering network usage and server load.
Lightweight Messaging: Messages are often sent in simple text or binary format, adding minimal overhead compared to multiple HTTP requests.

Next, let’s look at how WebSockets differ from traditional HTTP, this will help you understand how WebSockets are designed around HTTP protocol.

How WebSockets Differ from HTTP

Connection Lifecycle: HTTP is request-response based, meaning the client initiates every communication. In contrast, a WebSocket connection remains open, allowing the server to push updates at any time.
Protocol Upgrade: WebSockets start as an HTTP handshake, then upgrade the connection via the Upgrade header, switching to the WebSocket protocol ws or wss for secure connections).
Message Handling: Instead of sending headers with every request and response as in HTTP, WebSockets exchange lightweight packets (frames), which are efficient for real-time use cases.

Here we highlighted three important concepts which are the Connection Lifecycle which indicates that the connection for a WebSocket remains open for further updates when the traditional HTTP closes and new connections are initiated every time.

Establishing a WebSocket Connection

Initial HTTP Handshake: The client sends an HTTP request with a Upgrade: websocket header. The server responds with 101 Switching Protocols if it supports WebSockets.
Upgrading to WebSocket Protocol: After the server accepts, the connection switches from HTTP to the WebSocket protocol. Both ends maintain this upgraded connection until it’s closed by either party.
Bi-directional Messaging: With the connection established, the client and server can send messages to each other at any time. There’s no need for further handshakes until the connection is terminated.

const WebSocket = require('ws');
const wss = new WebSocket.Server({ port: 8080 });

wss.on('connection', (ws) => {
  console.log('New client connected');

  // Listen for incoming messages
  ws.on('message', (message) => {
    console.log(`Received: ${message}`);
    // Broadcast message to all clients
    wss.clients.forEach((client) => {
      if (client.readyState === WebSocket.OPEN) {
        client.send(message);
      }
    });
  });

  // Handle connection close
  ws.on('close', () => {
    console.log('Client disconnected');
  });
});

Use Cases for WebSockets

Chat Applications: Messaging apps like Slack or WhatsApp rely on immediate data flow, allowing users to receive new messages the moment they’re sent.
Live Sports Updates: Sports fans want to track scores in near real-time. WebSockets push these updates instantly, enhancing the viewing experience.
Financial Trading and Analytics: Stock price fluctuations are time-sensitive. WebSockets provide traders with instantaneous updates, which can be crucial for decision-making.
Online Multiplayer Games: Player actions and game states must be synchronized in real-time. WebSockets handle these rapid-fire events efficiently.

Designing Real-time APIs with WebSockets

Modern applications increasingly adopt WebSockets as a foundation for real-time APIs. This shift from conventional HTTP-based API design to WebSocket-driven architecture helps create APIs that are:

More Responsive: The server can immediately notify clients of new events.
Dynamic: Both sides can actively send data, fostering interactive experiences.
Efficient: Once established, the connection avoids the overhead of repeated HTTP requests, reducing latency and bandwidth usage.

Because of these advantages, WebSockets have become indispensable for any application that demands seamless, bi-directional communication.

Best Practices

Authentication: Ensure proper authentication before establishing a WebSocket connection. Tokens or cookies are commonly used to verify user identity during the initial handshake.
Error Handling: Handle connection errors gracefully, and implement reconnection logic on the client side for scenarios like network interruptions.
Scalability: As the number of concurrent connections grows, you’ll need strategies like load balancing and sticky sessions to direct users to the same server instance that maintains their WebSocket connection.
Security: Use secure WebSockets (wss://) in production. Always validate incoming data to protect against malicious payloads.
Monitoring and Logging: Track metrics like connection count, message throughput, and latency to detect performance issues or bottlenecks in real-time.

WebSockets are at the heart of real-time APIs, offering a faster and more efficient communication channel than traditional HTTP.

By maintaining a continuous, bi-directional connection, they excel in scenarios demanding instantaneous data transfer—such as chat applications, live sports updates, and real-time analytics.

This paradigm shift to a WebSocket-based design helps developers build APIs that are both responsive and capable of handling the high demands of today’s data-driven world.

If you’re building an application where timing and user engagement are paramount, WebSockets may just be the solution you’ve been looking for—empowering you to deliver dynamic, event-driven experiences that feel instantaneous, no matter how many users are online.

Did you learn any new things from this newsletter this week? Please reply to this email and let me know. Feedback like this encourages me to keep going.

See you on Next Week.

Remember to start learning backend engineering from our courses:

Get a 50% discount on any of these courses. Reach out to me (Reply to this mail)

Top 5 Remote Backend Jobs this week

Here are the top 5 Backend Jobs you can apply to now.

👨‍💻 BlueCat
✍️ Backend Engineer
📍Remote, Serbia
💰 Click on Apply for salary details
Click here to Apply for this role.

👨‍💻 Constructor
✍️ Backend Engineer: Experiments Team (Remote)
📍Remote, Worldwide
💰 Click on Apply for salary details
Click here to Apply for this role.

👨‍💻 Homepage
✍️ Backend Developer PHP
📍Remote, Worldwide
💰 Click on Apply for salary details
Click here to Apply for this role.

👨‍💻 Monzo
✍️ Backend Engineer
📍Remote
💰 Click on Apply for salary details
Click here to Apply for this role.

Want more Remote Backend Jobs? Visit GetBackendJobs.com

Backend Engineering Resources

Whenever you're ready

There are 4 ways I can help you become a great backend engineer:

2. The MB Academy: The “MB Academy” is a 6-month intensive Advanced Backend Engineering BootCamp to produce great backend engineers.

LAST WORD 👋

How am I doing?

Hit reply and say hello - I'd love to hear from you!

Stay awesome,
Solomon

I moved my newsletter from Substack to Beehiiv, and it's been an amazing journey. Start yours here.

API and API Design: What are Real-time APIs?

Solomom Eseme — Sat, 18 Jan 2025 17:02:34 +0000

Hello “👋

Welcome to another week, another opportunity to become a Great Backend Engineer.

Today’s issue is brought to you by Masteringbackend → A great resource for backend engineers. We offer next-level backend engineering training and exclusive resources.

Before we get started, I have a few announcements:

I have a special gift for you: You will love this one.

As developers, we are great at building products but selling them becomes a problem because “We hate marketing”?

For me:

I have been able to crack some of my marketing troubles, especially in the aspect of Copywriting following a book published by the founder of Vec Studio.

If you will someday or now enter into the world of CodePrenuers then this book is for you.

It’s 100% free

Click here to download the book

Drowning In Support Tickets? Maven AGI is here to help.

Maven AGI platform simplifies customer service by unifying systems, improving with every interaction, and automating up to 93% of responses. Seamlessly integrated with 50+ tools like Salesforce, Freshdesk, and Zendesk, Maven can deploy AI agents across multiple channels—text, email, web, voice, and apps—within days. Companies like Tripadvisor, ClickUp, and Rho slash response times by 60%, ensuring quicker support and exceptional customer satisfaction. Don’t let support tickets slow you down

Request a free personalized demo today.

Modern applications increasingly demand live updates and instantaneous communication. Whether it’s receiving stock market price changes or chatting with friends in a messaging app, Real-time APIs have become a critical part of delivering up-to-the-second information.

Imagine seeing a live match and there was a goal where Messi scored and your SportyBet app hasn’t updated it with your bet in it.

How will you feel?

Now that’s the importance of Real-time APIs and why you need to understand how it works and how to build one yourself as a backend engineer.

In this article, we’ll explore how Real-time APIs work, common use cases, architectural considerations, and some best practices for effective design.

What Are Real-time APIs?

At a high level, Real-time APIs are interfaces that push data to clients as soon as new information is available. Instead of sending a request and waiting for a response (as is typical in REST-based interactions), Real-time APIs keep the connection open, allowing the server to publish updates instantly.

There are key characteristics of Real-time APIs and these characteristics give birth to the different types of real-time APIs we will discuss later.

Persistent Connections: The client and server maintain a long-lived connection, often through WebSockets, Server-Sent Events (SSE), or other protocols that enable continuous communication.
Bi-directional or Uni-directional Flows: In some cases (e.g., WebSockets), both client and server can send messages at any time. In others (e.g., SSE), data flows primarily from the server to the client.
Event-driven Updates: The server broadcasts changes—such as a new chat message or a price update—as events that subscribed clients can act upon immediately.

Common Use Cases

I have already mentioned a few use cases. However, if your project falls under any of the categories listed below. Then you should consider using real-time APIs to build them.

Live Chat and Messaging Apps
Platforms like WhatsApp, Slack, or Microsoft Teams use Real-time APIs to deliver messages instantly to participants. As soon as a user sends a message, the server propagates it to all connected clients in real-time.
Financial Trading Platforms
Stock trading and cryptocurrency exchanges rely on real-time updates to track price fluctuations or order book changes. The difference of a few milliseconds can be crucial in high-frequency trading environments.
Online Multiplayer Games
Multiplayer games demand constant state synchronization among players. Real-time APIs handle position updates, score changes, and other game-state events to ensure a smooth experience.
Collaborative Tools
Document editing apps (like Google Docs) allow multiple users to edit the same file simultaneously. Real-time APIs power live updates, highlighting where others are editing and showing changes as they happen.

How Real-time APIs Work

Establishing a Persistent Connection

WebSockets: A full-duplex protocol allowing both the client and server to send data at any time. WebSockets begin as an HTTP handshake, then upgrade to a more efficient TCP-based protocol.
Server-Sent Events (SSE): A one-way channel from server to client. The client subscribes to a stream of events, and the server continuously sends updates as text-based messages.
Long Polling: Not truly a persistent connection, but an emulation of real-time behavior where the client sends a request that the server holds open until data is available.

We will discuss these types of real-time APIs in complete detail in the upcoming newsletters.

2. Data Transfer in Real Time

Once the connection is established, the server sends updates to the client whenever new data is available. If using a full-duplex protocol (e.g., WebSockets), the client can also send data to the server without initiating a new request each time.

3. Maintaining Connection State

Real-time APIs need to manage connections, handling scenarios such as:

Connection Loss: Detecting disconnections and re-establishing when the client comes back online.
Scaling: Balancing load across multiple servers while keeping messages consistent and ordered.

Example of Real-time Chat with WebSockets

Below is a simplified Node.js/Express server illustrating how a chat application might handle real-time communication using WebSockets:

const express = require('express');
const http = require('http');
const WebSocket = require('ws');

const app = express();
const server = http.createServer(app);
const wss = new WebSocket.Server({ server });

wss.on('connection', (ws) => {
  console.log('New client connected');

  ws.on('message', (message) => {
    console.log(`Received: ${message}`);
    // Broadcast to all clients
    wss.clients.forEach((client) => {
      if (client.readyState === WebSocket.OPEN) {
        client.send(message);
      }
    });
  });

  ws.on('close', () => {
    console.log('Client disconnected');
  });
});

server.listen(3000, () => {
  console.log('Server is listening on port 3000');
});

The code above uses the WebSocket server to create a connection and await the client’s messages after connection.

Real-time APIs are vital in scenarios where milliseconds matter—chat apps, financial dashboards, gaming, and collaborative platforms. By maintaining open connections and allowing continuous data exchange, they deliver a seamless, interactive user experience. However, designing such APIs requires careful attention to scalability, security, and connection management.

Did you learn any new things from this newsletter this week? Please reply to this email and let me know. Feedback like this encourages me to keep going.

See you on Next Week.

Remember to start learning backend engineering from our courses:

Get a 50% discount on any of these courses. Reach out to me (Reply to this mail)

Top 5 Remote Backend Jobs this week

Here are the top 5 Backend Jobs you can apply to now.

👨‍💻 BlueCat
✍️ Backend Engineer
📍Remote, Serbia
💰 Click on Apply for salary details
Click here to Apply for this role.

👨‍💻 Constructor
✍️ Backend Engineer: Experiments Team (Remote)
📍Remote, Worldwide
💰 Click on Apply for salary details
Click here to Apply for this role.

👨‍💻 Homepage
✍️ Backend Developer PHP
📍Remote, Worldwide
💰 Click on Apply for salary details
Click here to Apply for this role.

👨‍💻 Monzo
✍️ Backend Engineer
📍Remote
💰 Click on Apply for salary details
Click here to Apply for this role.

Want more Remote Backend Jobs? Visit GetBackendJobs.com

Backend Engineering Resources

Whenever you're ready

There are 4 ways I can help you become a great backend engineer:

2. The MB Academy: The “MB Academy” is a 6-month intensive Advanced Backend Engineering BootCamp to produce great backend engineers.

LAST WORD 👋

How am I doing?

Hit reply and say hello - I'd love to hear from you!

Stay awesome,
Solomon

I moved my newsletter from Substack to Beehiiv, and it's been an amazing journey. Start yours here.

Backend Engineers Welcome to 2025

Solomom Eseme — Tue, 07 Jan 2025 10:25:52 +0000

Hello “👋

Welcome to another week, another opportunity to become a Great Backend Engineer.

Today’s issue is brought to you by Masteringbackend → A great resource for backend engineers. We offer next-level backend engineering training and exclusive resources.

Hi everyone,

I want to use this medium to welcome old and new subscribers to this wonderful new year and also to outline the content direction we are taking to help you become a Great Backend Engineer.

First, thanks to everyone who stuck with us throughout last year and the new subscribers who joined us.

Below are some of the products, courses, and content we plan to release this year. As a subscriber to this newsletter, you will have the utmost benefits and huge discounts for any product or courses we share here.

Content Plan

We have already structured all the content that will be sent out this year. Here’s a summary of our content pipeline.

January → API Design
February → Web Security
March → Software Testing
April → Software Design Principles
May → Search Engines
June → Deployment
July → Software Caching
August → GraphQL
September → Microservices
October → Building a SaaS
November → Data Structures and Algorithms
December → Containerization and CI/CD

Click here for a detailed outline of each week’s content with the monthly content topic listed above.

Don’t enjoy all this wealth of knowledge alone this year.

Courses

We planned to complete and release 4 major courses with a detailed roadmap for becoming a Great Backend Engineer.

Become a Java + Spring Backend Engineer:

All-in-one Java and Spring course for learning backend engineering with Java. This comprehensive course is designed for Backend Engineers seeking proficiency in Backend Engineering with Java.

The course comes with:

10+ in-depth modules
50+ in-depth chapters
160+ high-quality lessons
48+ hours of video training content

The course is organized into 10 in-depth modules:

Java Essentials
Advanced Java
Building Backend Systems
Design Patterns in Java
Building REST APIs
Testing In Java
GraphQL with Java
Microservices in Java
Introduction to Kubernetes
Containerizing and Deploying Java Applications
Build Milestone Projects

We’ve released the first module and will be releasing modules every month to give you some time to practice properly.

Pre-order Now at 44% Discount

Become a Python Backend Engineer:

This comprehensive course is designed to guide you through the fundamental and advanced concepts of Python programming, progressing to practical skills for building scalable backend systems, implementing logging and caching, creating REST APIs, exploring GraphQL, testing strategies, containerization, deployment, and hands-on project development.

The course comes with:

10+ in-depth modules
50+ in-depth chapters
300+ high-quality lessons
48+ hours of video training content

The course is organized into 10 in-depth modules:

Python Essentials
Advanced Python
Building Backend Systems
Logging and Caching
Building REST APIs
Testing In Python
GraphQL with Python
Databases In Python
Containerizing and Deploying Python Applications
Build Milestone Projects

I also prepared two bonus mini-projects and bundled them inside. We will build 10+ Python-based projects to help you solidify your learnings.

Pre-order Now at 44% Discount

Become a Node.js Backend Engineer:

This comprehensive course is designed to guide you through the fundamental and advanced concepts of Node.js programming, progressing to practical skills for building scalable backend systems, implementing logging and caching, creating REST APIs, exploring GraphQL, testing strategies, containerization, deployment, and hands-on project development.

The course comes with:

10+ in-depth modules
50+ in-depth chapters
350+ high-quality lessons
48+ hours of video training content

The course is organized into 10 in-depth modules:

Node.js Essentials
Advanced Node.js
Building Backend Systems
Logging and Caching
Building REST APIs
Testing In Node.js
GraphQL with Node.js
Databases In Node.js
Containerizing and Deploying Node.js Applications
Build Milestone Projects

We’ve released the first module and will be releasing modules every month to give you some time to practice properly.

Pre-order Now at 44% Discount

Become a Rust Backend Engineer:

This comprehensive course is designed to equip you with the essential skills to master Rust programming, progressing to advanced concepts, testing strategies, web development, database connectivity, containerization, and deployment of Rust applications.

The course comes with:

10+ in-depth modules
50+ in-depth chapters
350+ high-quality lessons
48+ hours of video training content

The course is organized into 10 in-depth modules:

Rust Essentials
Advanced Rust
Web development with Rust
Actix-web framework
Building REST APIs
Testing In Rust
GraphQL with Rust
Databases In Rust
Containerizing and Deploying Rust Applications
Build Milestone Projects

We’ve released the first module and will be releasing modules every month to give you some time to practice properly.

Pre-order Now at 44% Discount

Projects

Below are some of the cool projects we are launching this year.

30RubyProjects: This is a challenge-based course where we challenge you to build 30 Ruby projects and land your dream Ruby Backend Role in 90 days.
30RustProjects: This is a challenge-based course where we challenge you to build 30 Ruby projects and land your dream Rust Backend Role in 90 days.
30PythonProjects: This is a challenge-based course where we challenge you to build 30 Ruby projects and land your dream Python Backend Role in 90 days.
30NodejsProjects: This is a challenge-based course where we challenge you to build 30 Ruby projects and land your dream Node.js Backend Role in 90 days.
30JavaProjects: This is a challenge-based course where we challenge you to build 30 Ruby projects and land your dream Node.js Backend Role in 90 days.

Each challenge route will cost $50 and we will work together including a community of like-minded folks to land you a backend role in 90 days. Each challenge will include Interview preparation kits, data structure, algorithm kits, and many more.

If you want to make the payment now, you can get a $30 flat fee for any route you take available only this week. Reply “Challenge Accepted” and I will send you a link to join the waitlist.

Did you learn any new things from this newsletter this week? Please reply to this email and let me know. Feedback like this encourages me to keep going.

See you on Next Week.

Remember to start learning backend engineering from our courses:

Backend Engineering Resources

Whenever you're ready

There are 4 ways I can help you become a great backend engineer:

2. The MB Academy: The “MB Academy” is a 6-month intensive Advanced Backend Engineering BootCamp to produce great backend engineers.

LAST WORD 👋

How am I doing?

Hit reply and say hello - I'd love to hear from you!

Stay awesome,
Solomon

I moved my newsletter from Substack to Beehiiv, and it's been an amazing journey. Start yours here.

API and API Design: How Do Simple JSON APIs Work?

Solomom Eseme — Wed, 04 Dec 2024 16:30:50 +0000

Hello “👋

Welcome to another week, another opportunity to become a Great Backend Engineer.

Today’s issue is brought to you by Masteringbackend → A great resource for backend engineers. We offer next-level backend engineering training and exclusive resources.

Before we get started, I have a few announcements:

I have a special gift for you: You will love this one.

Unlock Windsurf Editor, by Codeium.

Introducing the Windsurf Editor, the first agentic IDE. All the features you know and love from Codeium’s extensions plus new capabilities such as Cascade that act as collaborative AI agents, combining the best of copilot and agent systems. This flow state of working with AI creates a step-change in AI capability that results in truly magical moments.

Download It Free Today

Now, back to the business of today.

In my previous series, I explored everything you need to know and learn about API and API Designs: Different API Styles.

Today, we will discuss one API style called Simple JSON APIs, we will look at how Simple JSON APIs work and the different components of Simple JSON APIs.

This comes from my Backend Engineering Hub under the API and API Design section. However, I’m only transferring the knowledge here and breaking it down in this series one topic at a time.

Before we move on If you enjoy this newsletter. Invite your friends and get my books for free.

How Do Simple JSON APIs Work?

Simple JSON APIs have become a cornerstone of modern web and mobile app development in the digital age of connected applications.

They provide a lightweight, efficient, and intuitive way to exchange data between a client and a server. JSON (JavaScript Object Notation) plays a central role in this process, thanks to its simplicity and compatibility with various programming languages.

What is a JSON API?

A JSON API is an API that uses JSON as its data format for communication between a client (frontend) and a server (backend). JSON APIs facilitate seamless data exchange by structuring information into key-value pairs.

With its human-readable and lightweight structure, JSON is ideal for APIs designed to perform CRUD operations (Create, Read, Update, Delete) on data. Its versatility has made it the de facto standard for web APIs in various domains, from e-commerce to social media platforms.

How Do JSON APIs Work?

The workflow of a Simple JSON API can be broken down into three primary steps:

The client makes a Request

A client application (e.g., a web browser, mobile app, or IoT device) sends an HTTP request to the API server. The request specifies:

HTTP Method: Indicates the type of operation (GET, POST, PUT, DELETE, etc.).
Endpoint: The URL of the API resource being accessed.
Headers: Metadata, including content type (application/json) and authorization tokens (if required).
Payload (Body): Data sent in the request body (typically for POST or PUT methods).

Example Request

Here’s an HTTP POST request to create a new user:

POST /api/users HTTP/1.1  
Host: example.com  
Content-Type: application/json  

{  
  "name": "John Doe",  
  "email": "john.doe@example.com",  
  "password": "securePassword123"  
}

The server Processes the Request

The server processes the request in several steps:

Routing: The server determines the endpoint and matches it with the correct handler.
Validation: It validates the request data to adhere to the API's rules.
Database Interaction: The server interacts with the database (if needed) to perform the requested operation.
Response Creation: After processing, the server sends back a response in JSON format.

Example Backend Code (Node.js)

const express = require('express');
const app = express();

app.use(express.json());

// Endpoint to create a new user
app.post('/api/users', (req, res) => {
  const { name, email, password } = req.body;

  // Simulate saving user to a database
  const newUser = { id: 1, name, email };

  res.status(201).json({
    message: 'User created successfully',
    data: newUser,
  });
});

app.listen(3000, () => {
  console.log('Server running on http://localhost:3000');
});

Client Receives a Response

The server sends a response back to the client. The response contains:

HTTP Status Code: Indicates the outcome (e.g., 200 OK, 201 Created, 400 Bad Request).
Headers: Metadata such as Content-Type: application/json.
Body: A JSON payload with the result of the operation.

Example Response:

{
  "message": "User created successfully",
  "data": {
    "id": 1,
    "name": "John Doe",
    "email": "john.doe@example.com"
  }
}

The client processes this response and updates the UI or performs further actions as needed.

Why Are JSON APIs So Popular?

Lightweight and Fast: JSON’s compact structure reduces payload size, making APIs faster and more efficient.
Human-Readable: JSON’s syntax is easy to read, write, and debug.
Cross-Language Support: Virtually all programming languages support JSON parsing.
Flexibility: JSON APIs are versatile and can represent complex data structures like arrays and nested objects.
Widespread Adoption: JSON APIs are supported by modern tools, frameworks, and libraries, making them a go-to choice for developers.

Design Principles for Simple JSON APIs

Use Clear and Intuitive Endpoints: Design endpoints that clearly describe the resource they represent.

GET /api/users – Fetch all users.
GET /api/users/{id} – Fetch a specific user.

Consistent Use of HTTP Methods: Adopt standard HTTP methods for different operations:

GET: Retrieve data.
POST: Create new resources.
PUT/PATCH: Update resources.
DELETE: Remove resources.

Return Meaningful HTTP Status Codes: Always return appropriate status codes to reflect the result of the operation.

200 OK – Request succeeded.
201 Created – Resource successfully created.
400 Bad Request – Invalid request data.
401 Unauthorized – Missing or invalid authentication.

Structure JSON Responses Properly: Organize JSON responses to include metadata, data, and error information.

{
  "success": true,
  "data": {
    "id": 1,
    "name": "John Doe",
    "email": "john.doe@example.com"
  },
  "error": null
}

Secure the API: Use authentication (e.g., Token-Based Auth, OAuth) and HTTPS to protect data in transit.

Advantages of Simple JSON APIs

Ease of Integration: Simple JSON APIs are straightforward to implement and consume.
Scalability: They work seamlessly across devices, platforms, and networks.
Developer-Friendly: Widely supported by tools and frameworks, JSON APIs speed up development.
Reduced Overhead: Smaller payloads mean better performance, especially for mobile applications.

Simple JSON APIs are the backbone of modern application development. By leveraging JSON's lightweight, readable structure, these APIs enable seamless communication between clients and servers. Their simplicity, combined with robust design principles, makes them a preferred choice for developers building scalable and efficient web and mobile applications.

Whether you are developing a small app or a large enterprise system, mastering Simple JSON APIs is a key step toward creating robust and user-friendly solutions.

Did you learn any new things from this newsletter this week? Please reply to this email and let me know. Feedback like this encourages me to keep going.

See you on Next Week.

Remember to start learning backend engineering from our courses:

Get a 50% discount on any of these courses. Reach out to me (Reply to this mail)

Backend Engineering Resources

Whenever you're ready

There are 4 ways I can help you become a great backend engineer:

2. The MB Academy: The “MB Academy” is a 6-month intensive Advanced Backend Engineering BootCamp to produce great backend engineers.

LAST WORD 👋

How am I doing?

Hit reply and say hello - I'd love to hear from you!

Stay awesome,
Solomon

I moved my newsletter from Substack to Beehiiv, and it's been an amazing journey. Start yours here.

API and API Design: How Does SOAP Work?

Solomom Eseme — Sat, 30 Nov 2024 17:26:42 +0000

Hello “👋

Welcome to another week, another opportunity to become a Great Backend Engineer.

Today’s issue is brought to you by Masteringbackend → A great resource for backend engineers. We offer next-level backend engineering training and exclusive resources.

Before we get started, I have a few announcements:

I have a special gift for you: You will love this one.

Try the internet’s easiest File API

Tired of spending hours setting up file management systems? Pinata’s File API makes it effortless. With simple integration, you can add file uploads and retrieval to your app in minutes, allowing you to focus on building features instead of wasting time on unnecessary configurations. Our API provides fast, secure, and scalable file management without the hassle of maintaining infrastructure.

Try it now!

Now, back to the business of today.

In my previous series, I explored everything you need to know and learn about API and API Designs: Different API Styles.

Today, we will discuss one API style called SOAP, we will look at how SOAP works and the different components of SOAP.

This comes from my Backend Engineering Hub under the API and API Design section. However, I’m only transferring the knowledge here and breaking it down in this series one topic at a time.

How Does SOAP Work?

In the world of API communication, Simple Object Access Protocol (SOAP) stands as one of the most robust and structured methods for exchanging data.

Introduced in the late 1990s, SOAP remains a powerful choice for enterprises requiring reliable and standardized communication between systems.

But how does SOAP work, and what makes it suitable for specific use cases?

Let's explore:

What is SOAP?

SOAP is a protocol for exchanging structured information to implement web services. Unlike REST, an architectural style, SOAP is a strict protocol defined by standards from the World Wide Web Consortium (W3C).

SOAP messages are XML-based, ensuring platform-agnostic communication between applications.

SOAP is ideal for scenarios requiring high security, ACID-compliant transactions, or complex operations, such as banking or telecommunication services.

Key Components of SOAP

SOAP works through a combination of the following components:

1. SOAP Envelope

The SOAP Envelope is the container for the message. It defines the start and end of the message and includes metadata about the message.

2. SOAP Header

The Header is optional and contains metadata such as authentication credentials, transaction ID, or routing information.

3. SOAP Body

The Body holds the actual message or the data to be processed. This is where requests and responses are encapsulated.

4. WSDL (Web Services Description Language)

WSDL is an XML-based contract that describes the operations a SOAP service offers, including the parameters, return types, and endpoints.

How SOAP Works

Step 1: Client Sends a Request

The client generates a SOAP request message, encapsulated in an XML document. This message includes the operation to be performed and the necessary parameters


  
    
      exampleUser
      securePassword
    
  
  
    
      123

Step 2: Transmission via HTTP

The SOAP request is sent over HTTP or HTTPS to the server. Although HTTP is the most common transport protocol, SOAP can also work with SMTP, FTP, or other protocols.

Step 3: Server Processes the Request

On the server side:

The SOAP message is parsed to understand the requested operation.
The server performs the requested action using the provided parameters (e.g., fetching user data from a database).
The server generates a SOAP response.

Step 4: Server Sends a Response

The response is sent back to the client in the form of another SOAP message.

Example SOAP Response:


  
    
      
        123
        John Doe
        john.doe@example.com

The client then parses the response and processes the returned data.

Advantages of SOAP

Platform Independence: SOAP is language and platform-agnostic, thanks to its reliance on XML.
Standardized Protocol: It follows strict standards, making it highly interoperable.
Security: SOAP supports WS-Security for encryption and authentication, making it ideal for secure transactions.
Built-In Error Handling: SOAP messages include fault elements to describe errors in a structured manner.
Extensibility: The header can carry additional metadata for custom functionality.

Disadvantages of SOAP

Verbosity: XML messages are larger compared to JSON, leading to higher overhead.
Complexity: SOAP’s rigid standards and XML syntax can be cumbersome to work with.
Performance: The verbosity and processing requirements of XML make SOAP slower than lightweight protocols like REST.

How Does SOAP Work?

In the world of API communication, Simple Object Access Protocol (SOAP) stands as one of the most robust and structured methods for exchanging data. Introduced in the late 1990s, SOAP remains a powerful choice for enterprises requiring reliable and standardized communication between systems. But how does SOAP work, and what makes it suitable for specific use cases? Let's explore.

What is SOAP?

SOAP is ideal for scenarios requiring high security, ACID-compliant transactions, or complex operations, such as banking or telecommunication services.

Key Components of SOAP

SOAP works through a combination of the following components:

1. SOAP Envelope

The SOAP Envelope is the container for the message. It defines the start and end of the message and includes metadata about the message.

2. SOAP Header

The Header is optional and contains metadata such as authentication credentials, transaction ID, or routing information.

3. SOAP Body

The Body holds the actual message or the data to be processed. This is where requests and responses are encapsulated.

4. WSDL (Web Services Description Language)

WSDL is an XML-based contract describing the SOAP service's operations, including the parameters, return types, and endpoints.

How SOAP Works

Step 1: Client Sends a Request

The client generates a SOAP request message, encapsulated in an XML document. This message includes the operation to be performed and the necessary parameters.

Example SOAP Request:


  
    
      exampleUser
      securePassword
    
  
  
    
      123

Step 2: Transmission via HTTP

The SOAP request is sent over HTTP or HTTPS to the server. Although HTTP is the most common transport protocol, SOAP can also work with SMTP, FTP, or other protocols.

Step 3: Server Processes the Request

On the server side:

The SOAP message is parsed to understand the requested operation.
The server performs the requested action using the provided parameters (e.g., fetching user data from a database).
The server generates a SOAP response.

Step 4: Server Sends a Response

The response is sent back to the client in the form of another SOAP message.

Example SOAP Response:


  
    
      
        123
        John Doe
        john.doe@example.com

The client then parses the response and processes the returned data.

Advantages of SOAP

Platform Independence: SOAP is language and platform-agnostic, thanks to its reliance on XML.
Standardized Protocol: It follows strict standards, making it highly interoperable.
Security: SOAP supports WS-Security for encryption and authentication, making it ideal for secure transactions.
Built-In Error Handling: SOAP messages include fault elements to describe errors in a structured manner.
Extensibility: The header can carry additional metadata for custom functionality.

Disadvantages of SOAP

Verbosity: XML messages are larger compared to JSON, leading to higher overhead.
Complexity: SOAP’s rigid standards and XML syntax can be cumbersome to work with.
Performance: The verbosity and processing requirements of XML make SOAP slower than lightweight protocols like REST.

Use Cases for SOAP

Despite the rise of REST and GraphQL, SOAP remains a critical protocol in industries where security, reliability, and standardization are paramount.

Common Scenarios:

Financial Transactions: Banking APIs often use SOAP for secure, ACID-compliant operations.
Telecommunications: SOAP is used for provisioning services and managing billing systems.
Government Services: Standardization and security make SOAP a preferred choice for inter-governmental systems.
Enterprise Applications: ERP and CRM systems often integrate using SOAP.

Comparison: SOAP vs. REST

Feature	SOAP	REST
Protocol Type	Strict Protocol	Architectural Style
Data Format	XML	JSON, XML, etc.
Security	WS-Security	OAuth, Token-Based
Statefulness	Stateful or Stateless	Stateless
Ease of Use	Complex	Simple

SOAP continues to serve as a backbone for enterprise-level APIs where security, reliability, and interoperability are non-negotiable. Its strict protocol, built-in error handling, and extensibility make it a preferred choice in high-stakes industries, despite its verbosity and complexity.

While modern API developers often lean toward REST or GraphQL for flexibility, understanding how SOAP works equips engineers to maintain and integrate with legacy systems or secure, regulated environments.

By mastering SOAP, developers can bridge the gap between legacy systems and modern applications, ensuring robust communication and data integrity across platforms.

Did you learn any new things from this newsletter this week? Please reply to this email and let me know. Feedback like this encourages me to keep going.

See you on Next Week.

Remember to start learning backend engineering from our courses:

Backend Engineering Resources

Whenever you're ready

There are 4 ways I can help you become a great backend engineer:

2. The MB Academy: The “MB Academy” is a 6-month intensive Advanced Backend Engineering BootCamp to produce great backend engineers.

LAST WORD 👋

How am I doing?

Hit reply and say hello - I'd love to hear from you!

Stay awesome,
Solomon

I moved my newsletter from Substack to Beehiiv, and it's been an amazing journey. Start yours here.

API and API Design: How gRPC APIs Work?

Solomom Eseme — Thu, 28 Nov 2024 15:56:50 +0000

Hello “👋

Welcome to another week, another opportunity to become a Great Backend Engineer.

Today’s issue is brought to you by Masteringbackend → A great resource for backend engineers. We offer next-level backend engineering training and exclusive resources.

Before we get started, I have a few announcements:

I have a special gift for you: You will love this one.

Writer RAG tool: build production-ready RAG apps in minutes

RAG in just a few lines of code? We’ve launched a predefined RAG tool on our developer platform, making it easy to bring your data into a Knowledge Graph and interact with it with AI. With a single API call, writer LLMs will intelligently call the RAG tool to chat with your data.

Integrated into Writer’s full-stack platform, it eliminates the need for complex vendor RAG setups, making it quick to build scalable, highly accurate AI workflows just by passing a graph ID of your data as a parameter to your RAG tool.

Learn more about our production ready RAG tooling here.

Now, back to the business of today.

In my previous series, I explored everything you need to know and learn about API and API Designs: Different API Styles.

Today, we will discuss one API style called gRPC, we will look at how gRPC works and the different components of gRPC.

This comes from my Backend Engineering Hub under the API and API Design section. However, I’m only transferring the knowledge here and breaking it down in this series one topic at a time.

How Does gRPC Work?

In the realm of modern distributed systems, gRPC has emerged as a high-performance framework for building efficient and scalable APIs. Developed by Google, gRPC (short for gRPC Remote Procedure Call) builds on HTTP/2 and Protocol Buffers (Protobuf) to deliver low-latency communication, strong typing, and cross-platform interoperability.

We’ll dive into the inner workings of gRPC, explaining its components, benefits, and real-world applications.

What Is gRPC?

gRPC is an open-source, high-performance framework that enables seamless communication between client and server applications. Unlike REST, which revolves around resources and HTTP verbs, gRPC uses a Remote Procedure Call (RPC) model, allowing clients to invoke methods on a remote server as if they were local function calls.

Key features of gRPC include:

Efficient Serialization with Protocol Buffers.
Full-Duplex Streaming using HTTP/2.
Multi-language Support, enabling polyglot microservices.
Bidirectional Communication, making it ideal for real-time applications.

How Does gRPC Work?

To understand how gRPC works, let’s break it down into its core components and processes:

Install Required Packages

Before we start, install the necessary dependencies:

npm init -y
npm install @grpc/grpc-js @grpc/proto-loader

Defining the Service

At the heart of gRPC is the service definition, written in Protocol Buffers (Protobuf). Protobuf is a language-neutral and platform-independent mechanism for serializing structured data, enabling high-speed communication. The service definition specifies:

The methods available.
The request and response data types.

Here’s an example of a Protobuf definition for a simple user service:

syntax = "proto3";

service UserService {
  // Unary RPC
  rpc GetUser (UserRequest) returns (UserResponse);

  // Server-streaming RPC
  rpc ListUsers (EmptyRequest) returns (stream UserResponse);
}

message UserRequest {
  int32 id = 1;
}

message UserResponse {
  int32 id = 1;
  string name = 2;
  string email = 3;
}

message EmptyRequest {}

Implement the gRPC Server

Create a file named server.js:

const grpc = require('@grpc/grpc-js');
const protoLoader = require('@grpc/proto-loader');
const PROTO_PATH = './user.proto';

// Load the Protobuf file
const packageDefinition = protoLoader.loadSync(PROTO_PATH, {
  keepCase: true,
  longs: String,
  enums: String,
  defaults: true,
  oneofs: true,
});

const userProto = grpc.loadPackageDefinition(packageDefinition).UserService;

// Sample user data
const users = {
  1: { id: 1, name: 'Alice', email: 'alice@example.com' },
  2: { id: 2, name: 'Bob', email: 'bob@example.com' },
};

// Implement the GetUser RPC
function getUser(call, callback) {
  const user = users[call.request.id];
  if (user) {
    callback(null, user);
  } else {
    callback({
      code: grpc.status.NOT_FOUND,
      details: 'User not found',
    });
  }
}

// Implement the ListUsers RPC
function listUsers(call) {
  Object.values(users).forEach((user) => call.write(user));
  call.end();
}

// Start the gRPC server
function main() {
  const server = new grpc.Server();
  server.addService(userProto.service, { GetUser: getUser, ListUsers: listUsers });
  server.bindAsync('0.0.0.0:50051', grpc.ServerCredentials.createInsecure(), () => {
    console.log('Server running at http://0.0.0.0:50051');
    server.start();
  });
}

main();

Implement the gRPC Client

Create a file named client.js:

const grpc = require('@grpc/grpc-js');
const protoLoader = require('@grpc/proto-loader');
const PROTO_PATH = './user.proto';

// Load the Protobuf file
const packageDefinition = protoLoader.loadSync(PROTO_PATH, {
  keepCase: true,
  longs: String,
  enums: String,
  defaults: true,
  oneofs: true,
});
const userProto = grpc.loadPackageDefinition(packageDefinition).UserService;

function main() {
  const client = new userProto('localhost:50051', grpc.credentials.createInsecure());

  // Unary RPC: GetUser
  client.GetUser({ id: 1 }, (error, response) => {
    if (!error) {
      console.log('User:', response);
    } else {
      console.error('Error:', error.message);
    }
  });

  // Server-streaming RPC: ListUsers
  const call = client.ListUsers({});
  call.on('data', (user) => {
    console.log('User:', user);
  });
  call.on('end', () => {
    console.log('Finished receiving user list.');
  });
}

main();

Explanation of the Code

Here I will try to explain the code above here.

Server:
- Implements the GetUser method for retrieving a single user by ID.
- Implements the ListUsers method for streaming all users.
Client:
- Uses the GetUser method to fetch a single user.
- Uses the ListUsers method to stream multiple users.
Protocol Buffers:
- Defines the service and message structure in a compact and efficient format.
- Automatically generates bindings for the server and client.
HTTP/2:
- Enables efficient communication between the client and server.

Key Benefits of gRPC

High Performance: Protobuf serialization and HTTP/2 reduce payload size and latency, making gRPC faster than REST.
Strong Typing: Protobuf enforces strict typing, reducing runtime errors and improving data validation.
Cross-Language Interoperability: gRPC supports multiple languages, enabling seamless integration across diverse systems.
Streaming Capabilities: Full-duplex streaming makes gRPC ideal for real-time applications.
Code Generation: Automatic generation of client and server code accelerates development.

Real-World Applications of gRPC

Microservices Communication: gRPC excels in internal communication between microservices, where performance and efficiency are critical.
Mobile and IoT Applications: gRPC’s compact payloads and HTTP/2 support make it ideal for devices with limited bandwidth.
Real-Time Systems: Use cases like live chat, video streaming, and financial trading benefit from gRPC’s streaming capabilities.

gRPC revolutionizes how applications communicate by combining the power of Protocol Buffers, HTTP/2, and a robust RPC model. Its ability to handle diverse communication patterns, deliver high performance, and integrate across languages makes it a top choice for modern API design.

Did you learn any new things from this newsletter this week? Please reply to this email and let me know. Feedback like this encourages me to keep going.

See you on Next Week.

Remember to start learning backend engineering from our courses:

Backend Engineering Resources

Whenever you're ready

There are 4 ways I can help you become a great backend engineer:

2. The MB Academy: The “MB Academy” is a 6-month intensive Advanced Backend Engineering BootCamp to produce great backend engineers.

LAST WORD 👋

How am I doing?

Hit reply and say hello - I'd love to hear from you!

Stay awesome,
Solomon

I moved my newsletter from Substack to Beehiiv, and it's been an amazing journey. Start yours here.

API and API Design: How GraphQL APIs Work?

Solomom Eseme — Tue, 26 Nov 2024 16:14:07 +0000

Hello “👋

Welcome to another week, another opportunity to become a Great Backend Engineer.

Today’s issue is brought to you by Masteringbackend → A great resource for backend engineers. We offer next-level backend engineering training and exclusive resources.

Before we get started, I have a few announcements:

I have a special gift for you: You will love this one.

Unlock Windsurf Editor, by Codeium.

Download It Free Today

Now, back to the business of today.

In my previous series, I explored everything you need to know and learn about API and API Designs: Different API Styles

Today, we will discuss one API style called GraphQL, we will look at how GraphQL works and the different components of GraphQL.

This comes from my Backend Engineering Hub under the API and API Design section. However, I’m only transferring the knowledge here and breaking it down in this series one topic at a time.

How Does GraphQL Work?

In modern APIs, GraphQL has emerged as a powerful alternative to REST, offering flexibility, efficiency, and precise data retrieval. Developed by Facebook in 2012 and open-sourced in 2015, GraphQL addresses many limitations of traditional API design.

The Basics of GraphQL

At its core, GraphQL is a query language and runtime for APIs. Unlike REST, which exposes multiple endpoints for different resources, GraphQL consolidates everything into a single endpoint.

Clients send queries to this endpoint, specifying exactly what data they need, and the server responds with precisely that data. This eliminates the issues of overfetching (retrieving unnecessary data) and underfetching (requiring multiple requests to gather all needed data).

For example, in a REST API, fetching a user’s details and their posts might require two endpoints:

/users/:id
/users/:id/posts

In GraphQL, this can be achieved with a single query:

query {
  user(id: 1) {
    name
    email
    posts {
      title
      content
    }
  }
}

The server processes this query and returns only the requested fields in a structured JSON format:

{
  "data": {
    "user": {
      "name": "Jane Doe",
      "email": "jane.doe@example.com",
      "posts": [
        {
          "title": "GraphQL Basics",
          "content": "Understanding how GraphQL works..."
        },
        {
          "title": "Advanced GraphQL",
          "content": "Diving deeper into GraphQL queries..."
        }
      ]
    }
  }
}

The Anatomy of GraphQL

To understand how GraphQL works, it’s important to break it down into its key components:

Schema
Query
Resolvers
Mutations

Schema

The schema is the backbone of a GraphQL API. It defines the structure of the data available through the API, including the types of resources and their relationships. Think of it as a contract between the client and the server.

Example of a simple schema for a blog application:

type User {
  id: ID!
  name: String!
  email: String!
  posts: [Post]
}

type Post {
  id: ID!
  title: String!
  content: String!
  author: User
}

type Query {
  user(id: ID!): User
  posts: [Post]
}

The schema can be the same with your database table fields or any data you want to expose to the Frontend.

Query

Clients use GraphQL queries to request data. The query language is flexible and allows clients to specify exactly what fields they need.

Here’s an example of a Query:

query {
  posts {
    title
    author {
      name
    }
  }
}

This query is exactly what your Frontend engineer will send to the backend to retrieve data. From this query, it is clear that the Frontend is looking to retrieve a collection of posts with the title and the author's name.

The server returns only the requested fields:

{
  "data": {
    "posts": [
      {
        "title": "GraphQL Basics",
        "author": {
          "name": "Jane Doe"
        }
      },
      {
        "title": "Advanced GraphQL",
        "author": {
          "name": "John Smith"
        }
      }
    ]
  }
}

Resolvers

Resolvers are functions on the server side that fetch the actual data for the fields requested in the query. They connect the GraphQL API to the data sources, such as databases or other APIs. This is where your backend logic happens.

Example resolver for a user query:

const resolvers = {
  Query: {
    user: (parent, args, context) => {
      return context.db.getUserById(args.id);
    }
  }
};

The resolver above resolves the query when the Frontend requests a user detail. It finds the user via the ID specified and returns it to the Frontend as a response.

Mutations

While queries fetch data, mutations modify it. Mutations allow clients to perform operations like creating, updating, or deleting data. Inside the resolver explained above, you can also define a Mutation object to resolve all the mutations that will be created in your backend.

Here’s an example for your backend:

const resolvers = {
  Mutation: {
    addPost: (parent, args, context) => {
      return context.db.addPost(args.title, args.content);
    }
  }
};

Example mutation to add a new post:

mutation {
  addPost(title: "New GraphQL Post", content: "GraphQL is amazing!") {
    id
    title
  }
}

The server responds with the newly created post:

{
  "data": {
    "addPost": {
      "id": "101",
      "title": "New GraphQL Post"
    }
  }
}

Key Features and Benefits of GraphQL

Flexible Queries: Clients can request only the data they need, reducing overfetching and underfetching.
Single Endpoint: GraphQL APIs consolidate all resources into a single endpoint, simplifying client-side development.
Strong Typing: The schema provides a clear contract, ensuring clients and servers understand the data structure.
Real-Time Capabilities: With subscriptions, GraphQL can handle real-time updates, making it suitable for dynamic applications like chat apps or live dashboards.
Tooling and Ecosystem: GraphQL’s introspection capabilities enable powerful developer tools like GraphiQL and Apollo Client to test and interact with APIs.

Challenges of GraphQL

While GraphQL offers numerous advantages, it’s not without challenges:

Complexity in Implementation: Setting up resolvers and schema can be more complex than a REST API.
Overhead for Simple Use Cases: For straightforward APIs, GraphQL might be overkill.
Query Performance: Complex nested queries can lead to performance issues if not optimized.

GraphQL is revolutionizing the way APIs are designed and consumed, offering unparalleled flexibility and precision in data retrieval. Its ability to empower clients to shape their data needs makes it ideal for modern, dynamic applications. While it’s not a silver bullet, understanding how GraphQL works and when to use it can help developers build more efficient and scalable APIs.

Did you learn any new things from this newsletter this week? Please reply to this email and let me know. Feedback like this encourages me to keep going.

See you on Next Week.

Remember to start learning backend engineering from our courses:

Top 5 Remote Backend Jobs this week

Here are the top 5 Backend Jobs you can apply to now.

👨‍💻 Flying Bisons
✍️ Backend Developer
📍Remote
💰 Click on Apply for salary details
Click here to Apply for this role.

👨‍💻 Automattic Inc
✍️ Backend Software Engineer
📍Remote, Worldwide
💰 Click on Apply for salary details
Click here to Apply for this role.

👨‍💻 homepage
✍️ Backend Developer PHP
📍Remote, Worldwide
💰 Click on Apply for salary details
Click here to Apply for this role.

👨‍💻 Constructor
✍️ Backend Engineer: Search Features & APIs (Remote)
📍Remote
💰 Click on Apply for salary details
Click here to Apply for this role.

Want more Remote Backend Jobs? Visit GetBackendJobs.com

Backend Engineering Resources

Whenever you're ready

There are 4 ways I can help you become a great backend engineer:

2. The MB Academy: The “MB Academy” is a 6-month intensive Advanced Backend Engineering BootCamp to produce great backend engineers.

LAST WORD 👋

How am I doing?

Hit reply and say hello - I'd love to hear from you!

Stay awesome,
Solomon

I moved my newsletter from Substack to Beehiiv, and it's been an amazing journey. Start yours here.

Backend Weekly

API and API Design: API Security (part 2)

Looking for unbiased, fact-based news? Join 1440 today.

Broken Function Level Authorization

Example:

Solution:

Unrestricted Access to Sensitive Business Flows

Example:

Solution:

Server-Side Request Forgery

Example:

Solution:

Backend Engineering Resources

Whenever you're ready

API and API Design: API Security

Introduction

What is API Security?

Core Objectives of API Security

Common API Security Threats

Broken Authentication

Solution:

Excessive Data Exposure

Solution:

Lack of Rate Limiting

Solution:

Injection Attacks

Solution:

Key Principles of API Security

Backend Engineering Resources

Whenever you're ready

API & API Design: API Documentation Tools

You Don’t Need to Be Technical. Just Informed

Introduction

Why API Documentation Matters

What makes a Good Documentation Tool?

Popular API Documentation Tools

Swagger UI (OpenAPI)

DapperDox

ReDoc

Postman

How to Choose the Right Tool

Just to add:

Backend Engineering Resources

Whenever you're ready

API and API Design: API Keys & Management

Start learning AI in 2025

Introduction

What is an API Key?

Benefits of API Keys

What is API Management?

API Management Tools

Backend Engineering Resources

Whenever you're ready

API & API Design: Authorization Methods

Find out why 1M+ professionals read Superhuman AI daily.

Introduction

Authorization vs. Authentication

Role-Based Access Control (RBAC)

How it Works

Attribute-Based Access Control (ABAC)

How it Works

Conclusion

Backend Engineering Resources

Whenever you're ready

[New Podcast]: Rust + AWS Lambda: The Power of Serverless Performance

In this episode, we dive deep into:

Backend Engineering Resources

Whenever you're ready

API & API Design: Authentication Methods

Start learning AI in 2025

Introduction

Basic Authentication

API Key Authentication

Token-Based Authentication

JWT (JSON Web Tokens)

OAuth 2.0 Authentication

Session-Based Authentication

Comparing Authentication Methods

Conclusion

Backend Engineering Resources