14 min readaws

Tenant isolation on AWS for a multi-tenant SaaS

Filtering by tenant_id in application code is one line, in hundreds of places, and every one is load-bearing. This is how to move the boundary into IAM session tags and PostgreSQL row-level security, with a suite that deploys to AWS and tries to break every mechanism.

awssaassecurityiampostgresqlmulti-tenancy
Tenant isolation on AWS for a multi-tenant SaaS

Every multi-tenant SaaS I have worked on isolates tenants the same way:

php
Invoice::where('tenant_id', auth()->user()->tenant_id)->get();

This works until someone forgets it. It is one line, it appears in hundreds of places, and every one of them is load-bearing. Nothing about a missing filter looks wrong in code review. Nothing about it fails in testing, because in development there is only one tenant. It surfaces the first time a customer opens a page and sees somebody else's invoices, and by then it has probably been shipped for months.

The framework-level answers help and are worth using. Laravel global scopes, Rails default scopes, Django managers: all of them move the filter from every query to one place. But they are still application code, they are still bypassable by a raw query or a report builder or a background job that constructs its own connection, and they still fail open. Forget the scope and you get everything.

The alternative is to move the boundary underneath the application. Give a request credentials that cannot reach another tenant's data, and give the database a policy it applies to every query whether or not the application asked. Then a forgotten filter returns nothing rather than returning everything, and a SQL injection returns the attacker's own rows.

I built the whole thing on AWS, deployed it, and then wrote a suite whose job is to break it. Every check below is real output from that run. The code is at subeshb1/multi-tenant-isolation-aws.

None of this is language-specific, which is the point. The enforcement lives in Postgres and in IAM. The examples at the end are PHP because the project that prompted this was Laravel, but a Rails, Django, Go or Node application on the same infrastructure gets identical guarantees from the same two hooks.

Where the boundary can live

Tenant isolation gets discussed as three models, and they are usually presented as a choice. They are better understood as a spectrum of how much you are willing to pay.

Silo gives each tenant their own database, their own bucket, sometimes their own account. Isolation is perfect and the argument ends there. So does your unit economics: a hundred tenants is a hundred RDS instances, a hundred sets of backups, and a schema migration that is now a fleet operation. Real, and correct for a handful of enterprise customers who contractually require it.

Pool puts every tenant in the same tables, the same bucket, the same key. One migration, one instance, marginal cost per tenant close to zero. This is what almost every SaaS actually runs, and it is the model where isolation is a security problem rather than an accounting one.

Bridge is pool with a partition per tenant: a schema each in one database, a prefix each in one bucket.

The interesting question is not which model you pick, it is what enforces the boundary within the model you picked. In a pooled system the honest answer for most teams is "a where clause, applied by convention". Everything below is about replacing that with something a bug cannot switch off.

Four shared resources, and the four mechanisms that separate them

The stack holds every tenant's data in one S3 bucket, one DynamoDB table, one KMS key and one PostgreSQL database. Two tenants exist, acme and globex.

One IAM role for every tenant

There is a single TenantAccessRole. The tenant is not baked into it. It arrives as a session tag at AssumeRole time, and the policies read it back out:

json
{
  "Effect": "Allow",
  "Action": ["s3:GetObject", "s3:PutObject", "s3:DeleteObject"],
  "Resource": "arn:aws:s3:::tenant-data/${aws:PrincipalTag/tenant}/*"
}

Ten tenants or ten thousand, that is the same role and the same policy. The alternative, a role per tenant, runs into the roles-per-account quota (1,000 by default, adjustable) and turns customer onboarding into an IAM deployment.

Vending scoped credentials is three lines:

python
sts.assume_role(
    RoleArn=TENANT_ROLE_ARN,
    RoleSessionName=f"tenant-{tenant}",
    Tags=[{"Key": "tenant", "Value": tenant}],
)

The security of the entire scheme reduces to one question: who may call that, and can they choose the tenant argument? In a real system it sits behind authentication and the value comes from the validated token, never from anything in the request. A tenant id read out of a header, a query parameter or a subdomain is an authorization decision you have handed to the attacker.

One thing that catches people: assuming a role and tagging a session are two different actions. sts:TagSession has to be granted in the trust policy and in the caller's policy. Without it the tag never lands, ${aws:PrincipalTag/tenant} never resolves to anything, and the session can reach nothing at all. That is the correct failure, but it does not look like a missing permission, it looks like a broken policy.

S3 needs two statements, and the second one is the one people skip

Object access is scoped by putting the tag inside the resource ARN, which the policy above does. Listing is an action on the bucket, not on an object, so its ARN cannot carry the tenant and the scoping has to come from a condition instead:

json
{
  "Effect": "Allow",
  "Action": ["s3:ListBucket"],
  "Resource": "arn:aws:s3:::tenant-data",
  "Condition": { "StringLike": { "s3:prefix": ["${aws:PrincipalTag/tenant}/*"] } }
}

Grant ListBucket without that condition and a tenant cannot read anybody else's files but can enumerate every filename in your system. For a lot of products the filenames are the leak: customer names, deal names, patient identifiers, invoice numbers.

Running against the deployed bucket, with credentials vended for acme:

[  held  ] each tenant can write inside its own prefix
[  held  ] each tenant can read its own object
[  held  ] acme cannot GET globex's object
           AccessDenied: User: arn:aws:sts::123456789012:assumed-role/
           TenantAccessRole/tenant-acme is not authorized to perform: s3:GetObject
[  held  ] acme can list its own prefix
           1 object(s)
[  held  ] acme cannot list globex's prefix
           AccessDenied: ... is not authorized to perform: s3:ListBucket
[  held  ] acme cannot list the whole bucket
           AccessDenied: ... is not authorized to perform: s3:ListBucket

DynamoDB, and the action you must not grant

dynamodb:LeadingKeys constrains the partition key of every item a request touches:

json
"Condition": {
  "ForAllValues:StringEquals": {
    "dynamodb:LeadingKeys": ["${aws:PrincipalTag/tenant}"]
  }
}

What matters as much is what the policy leaves out. There is no dynamodb:Scan in it, and there must not be. A scan reads across partitions by definition, so LeadingKeys has nothing to constrain and the condition simply does not apply. If a tenant session can scan the table it can read every tenant, condition or no condition.

[  held  ] acme can query its own partition
           1 item(s)
[  held  ] acme cannot query globex's partition
           AccessDeniedException: ... not authorized to perform: dynamodb:Query
[  held  ] acme cannot Scan the table
           AccessDeniedException: ... not authorized to perform: dynamodb:Scan

KMS encryption context, which holds even when IAM does not

Each tenant's data is encrypted under the shared key with an encryption context of {"tenant": "acme"}, and both the key policy and the role policy pin that context to the principal tag:

json
"Condition": {
  "StringEquals": { "kms:EncryptionContext:tenant": "${aws:PrincipalTag/tenant}" }
}

The obvious test is that acme cannot decrypt globex's ciphertext under globex's context, and it cannot: IAM refuses. The more interesting test is what happens when it lies about the context to get past IAM.

[  held  ] acme cannot decrypt globex's ciphertext
           AccessDeniedException: ... not authorized to perform: kms:Decrypt
[  held  ] acme cannot decrypt it by claiming its own context
           InvalidCiphertextException: (no message) -- this one is not IAM.
           The policy allowed the call; the ciphertext failed to authenticate
           under the wrong context.

Note the different exception. The second call passed IAM cleanly, because the context now matches the principal tag. It failed in the cryptography instead. An encryption context is authenticated additional data, bound into the ciphertext when it is created, so the AEAD tag does not verify under a different context and KMS has nothing to say about it.

That is two independent layers, and the second one does not depend on your policy being right. If someone later widens the KMS policy by accident, a tenant holding another tenant's ciphertext still cannot read it. This is the only mechanism here with that property, and it is why per-tenant encryption context is worth the trouble even when the IAM story is already good.

PostgreSQL row-level security, which is the one that matters

For a shared-database SaaS this is the mechanism that replaces the where clause, so it is worth getting exactly right.

sql
ALTER TABLE invoices ENABLE ROW LEVEL SECURITY;
ALTER TABLE invoices FORCE  ROW LEVEL SECURITY;

CREATE POLICY tenant_isolation ON invoices
    USING      (tenant_id = current_setting('app.tenant_id', true))
    WITH CHECK (tenant_id = current_setting('app.tenant_id', true));

Four details in those four lines carry all the weight.

FORCE, not just ENABLE. ENABLE turns policies on for everyone except the table owner. If the application connects as the owner, and it usually does because that is the user the migrations ran as, the policies are ignored entirely and everything looks like it is working. The proof suite has two identical tables to make this concrete, one forced and one not, with the same policy and the same seven rows:

[  held  ] FORCE: the table owner is subject to the policy
           invoices: 3 rows visible to the owner (only acme's)
[  held  ] no FORCE: the table owner sees every tenant
           invoices_unforced: 7 rows visible to the owner, all tenants.
           Same policy, same query, one missing keyword.

Three rows against seven. Same policy, same query, one missing keyword. If you take one thing from this post, take that the role your application logs in as must not own the tables.

WITH CHECK, not just USING. USING filters what a query can see. WITH CHECK filters what a write is allowed to produce. Without it a tenant reads only its own rows and writes rows labelled with anybody's tenant_id, which is a data-poisoning primitive rather than a read leak, and considerably harder to notice.

The second argument to current_setting. current_setting('app.tenant_id', true) returns NULL instead of raising when the setting is absent. tenant_id = NULL is NULL, which is not true, so a connection that never set a tenant sees nothing. The failure mode is an empty result set, not a full table. Fail closed.

The application's role owns nothing and is NOBYPASSRLS. Three separate identities: the master user runs migrations, app_owner owns the tables and cannot log in, app_rw is what the application connects as and owns nothing.

Here is the whole database section from the run:

[  held  ] an unfiltered SELECT as acme returns only acme rows
           3 rows, all tenant_id=acme; the query had no WHERE clause
[  held  ] a tautology in the WHERE clause changes nothing
           3 rows, still only acme
[  held  ] asking explicitly for globex returns nothing
           the policy is ANDed into the query plan, so this is not reachable
[  held  ] acme cannot insert a row labelled globex
           new row violates row-level security policy for table "invoices"
[  held  ] a connection with no tenant set sees zero rows
           count = 0
[  held  ] the app role cannot disable row-level security
           must be owner of table invoices
[  held  ] the app role cannot become the table owner
           permission denied to set role "app_owner"
[  held  ] the app role cannot drop the policy
           must be owner of relation invoices

The second check is the one worth dwelling on. SELECT tenant_id FROM invoices WHERE 1=1 OR TRUE is the shape a SQL injection actually takes, and it is also the shape a broken query builder takes. It returns three rows, because the policy is ANDed into the plan by the planner, below anything the query text can influence. There is no clause you can write that reaches the other four rows.

The last three checks matter for a different reason. They are the difference between an attacker who has achieved SQL execution reading one tenant and reading all of them. app_rw cannot turn the policy off, cannot escalate to the owner, and cannot drop the policy, so compromising the application does not compromise the boundary.

The bug this ships as: SET instead of SET LOCAL

Everything above is correct and will still leak if the tenant is set at session level.

[  note  ] a session-level SET is still there on the next request
           request 1 set tenant_id='acme' and saw 3 rows. Request 2 set
           nothing and read tenant_id='acme', 3 rows. Row-level security did
           exactly what it was told; it was told the wrong tenant.

Read that carefully, because it is not an RLS failure. The policy worked perfectly. The connection came back out of the pool still carrying the previous request's tenant, and the next request, belonging to a different customer, inherited it. Every query it ran was correctly scoped to somebody else.

This is worse in exactly the environments people reach for to make Laravel fast. Under Octane the process and its connections persist between requests by design. Behind PgBouncer in transaction mode, connection reuse across clients is the entire feature. Both of them turn a working policy into a cross-tenant leak, and neither shows up in a test suite, because tests do not hand one connection to two different users.

SET LOCAL is scoped to the transaction, so the request has to run inside one:

[  held  ] SET LOCAL is gone at COMMIT, so nothing can be inherited
           after the transaction the setting is '' and the same query returns
           0 rows. The next request starts with no tenant, and no tenant
           means no rows.

In Laravel that is a middleware:

php
public function handle(Request $request, Closure $next): Response
{
    // From the authenticated session. Never from a header, a query
    // parameter or a subdomain: those are the attacker's to choose.
    $tenant = $request->user()?->tenant_id;

    abort_if($tenant === null, 403, 'No tenant on this request');

    return DB::transaction(function () use ($tenant, $request, $next) {
        DB::statement("SELECT set_config('app.tenant_id', ?, true)", [$tenant]);

        return $next($request);
    });
}

Wrapping the request in a transaction is not incidental, it is what gives SET LOCAL something to be local to. Everything downstream can then write Invoice::all() and get only the current tenant's invoices. No global scope to remember, no trait to add to a model, no where('tenant_id', ...) for a reviewer to notice is missing.

Do the same in your queue jobs. Jobs run outside the HTTP middleware stack, they are where "we scoped everything" quietly stops being true, and a job that runs with no tenant set now returns zero rows instead of processing everybody's.

Running the proofs yourself

The suite runs as a one-off Fargate task in the same account. The task holds no permissions of its own beyond the right to assume the tenant role, which is precisely the position your application server is in.

bash
export AWS_REGION=us-west-2
./deploy/bootstrap-secrets.sh
cd infra && npm install && npx cdk deploy
cd .. && ./deploy/run-proofs.sh
Result
------
26/26 isolation checks held
No cross-tenant access was possible.

Exit code is 0 only if every check held, so this belongs in CI. That is the real argument for writing the suite at all: isolation is not a property you establish once, it is a property that a policy edit six months from now can silently remove. A test that tries to read the other tenant and expects to fail is the only thing that notices.

Two things I tripped over deploying this. isolation is a reserved word for the RDS PostgreSQL engine, so databaseName: 'isolation' fails the create ten minutes into a deploy, after the instance has already spent that long provisioning. And the RDS master user cannot DROP OWNED BY a role it is not a member of:

ERROR: role "app_rw" cannot be dropped because some objects depend on it
DETAIL: Only roles with privileges of role "app_rw" may drop objects owned by it.

which is the same principle the rest of the setup relies on, arriving from the other direction: the master user is not a superuser on RDS and does not get privileges it was not given. The idempotent teardown in schema.sql grants itself membership first.

What this does not cover

Noisy neighbours. Isolation here is about access, not about one tenant's report query consuming the whole database. That is statement_timeout per role, connection limits, and eventually moving your largest customers to their own instance.

Per-tenant keys. One KMS key with an encryption context per tenant is the pooled answer. A tenant who contractually requires their own key, or the ability to revoke access by deleting a key, needs a key each. That is a real cost, $1 per key per month plus request charges, and worth it only when someone is asking.

Backups and analytics. A pg_dump runs as a superuser and ignores row-level security. So does the replica feeding your data warehouse, and so does every ETL job. The boundary you built for the application does not extend to the pipelines around it unless you extend it deliberately.

The vending machine itself. Everything here reduces to whether the caller of AssumeRole can choose the tenant argument. All the policies in the world do not help if that value comes from a request header. That code is the highest-value thing to review in the whole system, and it is about fifteen lines.

The point

Application-level tenant filtering is a control that fails open, applied by convention, in hundreds of places. Infrastructure-level isolation is a control that fails closed, applied once, that a bug in your application cannot switch off.

You still write the where clause, for query plans and for clarity. The difference is what happens when you forget.

The complete project, including the CDK stack, the SQL, the Laravel middleware and the full proof suite, is at github.com/subeshb1/multi-tenant-isolation-aws. If you are building or auditing a multi-tenant SaaS on AWS and want a hand, get in touch.

SB

Subesh Bhandari

Engineer · Writer · Builder

Join the conversation

Comments are powered by GitHub.