We just closed a major hole in our security that I wanted to cover here, because I cannot recall seeing a comprehensive discussion of this type of problem (or we would not have been hit by it).

The problem

We shifted all credentials for back-end services to environmental variables some time ago, in accordance with best practices. In the case of our APIs, which run on Express.js. this meant:

• Storing values in /var/www/html/[docRoot]/js/.env
• Exposing the values via a line in app.js:
  require( "dotenv" ).config( { path: __dirname + "/.env" } );
• Using the values internal to the routes as process.env.secretValue

What we missed (and what was not discussed and/or taken for granted by the authors of the sources we used) was that our secrets were wide open. You could go to the sites in question and pull a directory of all files (error 1), and, much worse, you could open our .env file in a web browser — exposing all of our credentials.

We were ignorant of this problem, and eventually someone came along, cracked open one of our files, and used our Sendgrid credentials to start spamming people using our service.

The solution

What we did was lock down the entire server by editing /etc/httpd/conf/httpd.conf which is the master policy record for the entire Apache server. In this file, we added two sections:

The above (colored in red) removed access to see directories across the entire server (we have no use-case to provide this). Also, in the same file, we removed all visibility on all .env files (thus hiding our credentials).

Having closed these holes, we still had to regenerate new credentials for all of our services, but now these secrets are relatively safe. While this content may seem obvious to many readers, the fact is these things were not obvious to us, and we are probably not alone, so we are admitting our error in public for the benefit of all.