This is my second try at a software blog. My first attempt was in college when I was studying at Wesleyan University, a small liberal arts college in Connecticut. At Wesleyan, the CS curriculum focused on math, proofs, and programming languages. I learned no practical software engineering skills. I didn't know what a unit test was, I hadn't heard of SQL, I hadn't heard of AuthN/AuthZ, and I didn't understand the basics of a client server relationship.
I also didn't have interview-style algorithms skills, so while applying for internships, I failed every automated assessment. This began my quest to learn practical skills and get a job. I joined the privacy tech lab at Wesleyan where I worked on Privacy Pioneer, a browser extension that intercepted HTTP requests and responses to generate "privacy labels" based on automated scanning techniques. We wrote a React app to implement the extension's pop-up window and webpage. The backend was a "service worker"—a small JavaScript program running in the background. Only Firefox has the `webRequest` API Privacy Pioneer needed to read HTTP requests and responses, so we never made it to Chrome.
What I didn't put together working on Privacy Pioneer was how to build software using managed services. I hadn't seen modeling services, deploying services, providing credentials to services, or instantiating clients to call services. Whether it's services for storage, for compute, for networking, for authentication-this is how most software is implemented. At AWS, most managed services are created by combining other AWS managed services!
In 3 years at AWS, without asking for any transfers, I've been shuffled to four teams and three orgs. Primarily I've worked in developer tooling: CodeBuild, Codecatalyst (failed GitHub competitor), Q Developer (llm chat panel in the AWS console), and now Bedrock AgentCore, which is my first departure from developer tooling. I've learned a lot in these three years, but it's still early days for me.
I'm hoping to use this blog to gather, reinforce, and remember my learnings in software. So, to inaugurate the blog, here's what I learned while making this website.
I went out of my way to make this website somewhat manually, but it's still completely dependent on managed services and existing libraries just "lower-level" ones. I didn't use anything fancy like Neflify, Vercel, CloudFront, CloudFlare Pages, GitHub Pages (what I used last time), or a no-code builder. My goal was to encounter the details that are abstracted by popular modern solutions.
Making this website was a good entrypoint towards networking. I finally wrapped my head around DNS through its full life-cycle: from client device -> router -> recursive resolver -> root DNS -> top level domain DNS server -> eventual resolution until resolver gets an A/AAAA record. I learned the easter egg that the recursive resolver starts with a root hints file to identify the IPs of the root DNS (so many systems need a gotcha invariant like this). The TLD DNS for .com is just a handful of DNS servers maintained by a company called Verisign. When you buy a .com domain, whether through CloudFlare or GoDaddy or Namecheap, a record is written into Verisign's servers pointing to the DNS that the chosen managed provider runs for you.
Similarly, on the topic of IP, when setting up an EC2 instance, you need to define the "security group" for which IPs can connect to the EC2. Security groups are relative to protocols which are relative to ports. Since this website serves HTTPS and redirects HTTP to HTTPS, I leave 0.0.0.0/0 open on port 80 and 443. For the SSH security group on port 22, the best practice is to /not/ be open to all internet traffic. But, since your ISP can change your router's public IP at any time, it's hard to provide a fixed IP.
The lazy solution to configuring a SSH security group is to leave port 22 open to all IPs, but to only accept key based-not password-auth. Mathematically, with key-only authentication, an open ssh port is not susceptible to brute force attack due to the mathematical properties of Ed25519 which is the default key signature system for EC2.
The search space for the security target (how much work to break the key) of 2^128 is so large, that even a quantum computer significantly more powerful than the SOTA is not even close to brute forcing a Ed25519 key. There are other exploits besides breaking the private key which make leaving port 22 open to all traffic not suited for important systems.
I've worked on AWS teams that configure services inside of a VPC and even updated Route53 records, but these experiences didn't force me to build a complete understanding. It's easy to finish a task by following existing patterns.
Nginx was popular in the Apache days as a full server solution. That's why it has nice functionality to serve static assets (which is what serves this website!). Nowadays, it's commonly used as a reverse-proxy in front of backend servers. The name "reverse-proxy" was always confusing to me, but all it means is:
"an entrypoint server running in front of the backend server/s that maintains connections with clients and opens new connections with the backend where application logic is implemented." It's the front man. Which is why the name "reverse" is confusing!
Services can expose the reverse-proxy IP and then clients don't become privy to any details about the backend networking configuration. To use nginx as an L7 (application layer) load balancer, point the DNS record at the IP running nginx and then inside of that server, make connections to backend services. Indeed, any L7 LB is also a reverse-proxy because it terminates TLS and routes requests by establishing new connections with backend services. Popular cloud-based ones would be AWS ALB, AWS API Gateway, and the GCP/Azure versions of these products.
To serve this website:
I started this first post using straight HTML and got stuck on a few <a></a> tags, so I tried to setup a markdown -> html converter
because nginx didn't have this out of the box. I picked a tool, pandoc, discovered that it's not installable on the AL2023 OS running on my EC2, and implemented a work-around where the GitHub runner
installs and runs pandoc itself, and then copies the final output into the EC2.
I looked back at this a week later and decided this was too complicated for the purpose of the website—serve static text. So I deleted it all! At the same time, I just spent an embarrassing amount of time fighting with HTML. Hopefully LLMs can bridge the tradeoff here and fix my HTML for me :)