In the first 6 months of my first job, fresh out of university, I had written a Javascript framework. As you do. This is a brief retelling of that story.

Background

We were a small company, with only 5 developers at the time. Despite that, we still managed to do a lot. At the time we were building a custom dashboard for our client’s product, which would give their users a better understanding of what work needed to be done. The base of the dashboard was a boilerplate Angular app, but since nobody actually knew how to use Angular (I must include myself in this) it had all sorts of regular Javascript pieces glued on the side. I found the codebase near impossible to work in. My co-workers found it near impossible to work in. And then, barely two weeks into my employment, I was flown to Melbourne to work in the clients’ office.

While there, I worked in their application; away from the troubles back home. But my mind was formulating a plan. If I could write a little framework, then I could slowly do away with the Angular app that was confusing the whole thing, and start writing code that could be read and understood. As I was learning and understanding the client’s system, my brain was also focused on something else: the core parts of a framework that would define the next year or two of my professional life. But, I needed to sell it to my boss. Luckily for me he was a technology guy, always looking for the next thing. He was also working in the clients’ office, so a few days before I was due to head back to New Zealand I asked if I could talk to him about an idea.

My pitch focused on two ideas:

1. Saving network requests Two of the widgets on the dashboard used the exact same data, but since they were separate in the code they were each making the network request. This was an expensive call, that ended up multiplying into multiple requests to our clients’ system. At times it would take so long to do that request it would just time out and the dashboard wouldn’t load properly. This was a pain point of my boss, and I knew that I could sell him on this idea alone.

2. Individually deployable widgets Each widget on the dashboard, once converted to this new framework, could be deployed on its own, without needing to touch any other components. The word “microfrontend” existed at the time, but we hadn’t heard of it so “individually deployable widgets” was the term we used. My boss liked new shiny technologies, and I knew that he wanted to have each widget split out anyway, so this framework could be the path forward to doing that. (Did we end up using this? No, of course not. It was far more convenient to have a single deployment process. We simply didn’t have the scale to require it.)

There was also the third reason I didn’t mention, was that I really didn’t like the code at that point and I wanted to rewrite it. I think my boss may have been able to see through me, but he said that when I got back I’d be able to work on this new framework for one sprint and see where I got from there.

I even had a name for it: The Thin Controller.

1. It controls the widgets.

2. “Thin” meant it was going to stay small and not bloat out into a big blob of 4000 lines of code that only I knew how to maintain, right?

3. It’s a reference to Thomas the Tank Engine, and jokes about trains were a thing since one of my co-workers was interested in them.

Getting started

With an official go-ahead, I set out. I knew I needed to fulfil those two goals of saving network and separate components, or else the project would have to be abandoned. In addition, anything using the Thin Controller must peacefully co-exist with the existing Angular/mish-mash application.

It was really refreshing to be able to go off on a code adventure with no oversight. I still attended the team’s standups to give progress updates, but for that brief time I was able to do what I wanted. And what I wanted was to build this framework. It took about a week to have the first proof of concept, which didn’t take much massaging to get to its first official version.

Saving network requests

The first thing I tackled was the one I found more fun: the network deduplication. I had finally understood how Javascript’s Promises worked, and this was a perfect opportunity to test that knowledge.

The concept was relatively simple:

• When a new request comes through the Controller, check to see if it is identical to a request that is currently in flight

  • If it is, then re-use that existing request and return a Promise for its response

  • If not, send off a new request, store it in the list of in-flight requests, and return a Promise for the response. This wouldn’t have any race conditions, due to Javascript’s single threaded nature.

• When a request finishes, remove it from the in-flight list and resolve all of the relevant Promises (which was really just passing it down the chain)

With a solid plan in mind, it didn’t take long to implement. I probably spent more time with the authentication side of the requests, since I needed to port that from the Angular app, than with this new component of the framework: now called the Traffic Manager.

The Traffic Manager stayed in the Controller for its entire life, and ended up being the least-touched part of the framework because it just worked as intended. Implementing this deduplication probably gave us a couple of months to sort other things out before we had to tackle the issue of slow requests again, since now the backend only had to deal with one slow request instead of two identical ones.

Individually deployable widgets

There was the matter of being able to deploy widgets without needing to deploy the whole frontend.

The approach I took here was a simple one: put each widget in their own folder on the web server. If you needed to deploy a widget, build it and copy it to its folder. Simple and easy to do.

Each widget’s code consisted of three files: and HTML fragment, any relevant CSS, and a Javascript file that had a class that extended Controller.Widget. The Widget base class had a few useful members, and would serve as a widget’s main way of interacting with the rest of the Controller.

Later versions would allow widgets to be in different places or even entirely separate web servers through the use of a manifest file, though this functionality was never used in practice.

Instead I’m going to take this part of the article to talk about the widget lifecycle. Upon loading, the Thin Controller would look for elements with a data-widget-type attribute. These would be set to the IDs of each widget. It would then load the three files for those widgets using the ID in the path. The CSS would be added to the page, and the innerHTML of the container element would be set to the HTML fragment. Finally, a new instance of the Widget’s class would be constructed, getting passed a reference to its container element.

This system allowed multiple widgets to be loaded in parallel, as well as the ability to have multiple instances of the same widget at the same time (though this was never used).

What was it like to use?

4/10. Better than having no framework, but I wouldn’t recommend it to anyone.

All the Controller did was provide a standard lifecycle with more streamlined loading of code, and basic networking so you didn’t have to handle the authentication yourself. Everything else was hands-off and left up to you. For the most part, this means implementing each widget as vanilla Javascript.

Later on, a designer at our client had access to our repo (as you do…) and would work on designs and basic UI interactions in our code. We ended up including jQuery in the main page, which they made heavy use of. Even later, I would use the Controller to load React apps wrapped up in the Widget class. The Controller didn’t care how you implemented something, which was simultaneously liberating and dangerous as it was easy to get something wrong.

The Controller was definitely built to fulfil its needs, and would not be suitable pretty much anywhere else. It started off being a more standard glue for our disparate components, but ended up being the foundation everything else was built upon. I suppose that was really its goal all along.

What was is like to write?

I had a solid plan going into it, and I was able to follow those ideas. That period of time is blurred in my mind, and tarred with the crunch we were feeling at the time, but I remember feeling a rush of freedom; like I was able to do what I felt needed to be done.

It’s a largely creative process, with lots of thought needing to be put into how the framework is going to be used. I knew it would be far more important to make the barrier of entry low, and the chances of getting something wrong low as well, in order for it to be a success. The specifics of the methods available on the Widget class went through a couple of iterations as I started working on one of the widgets.

With my knowledge of writing the framework, I was able to take some of the simpler widgets and convert them to Thin Controller widgets within that first sprint. Having actual visible artefacts on the page was a big help in making sure the Controller made it into the codebase. With my boss happy that I’d managed to combine those network requests and separate widgets out from the rest of the code, the first version of the Controller went to production.

Further improvements made over time

Things moved pretty fast at that company. I started work on the Controller in March, but by August the scope of what our frontend needed to do had grown so much that the Controller itself was no longer so thin. My time to work on the Controller was limited, so often improvements were made while working on other features.

Pages

The controller existed peacefully for all of a couple of months before more changes were required. Our client was looking for alternate streams of revenue, and we got tasked with creating a streamlined and integrated storefront experience right next to our dashboard. Pages sound a bit like widgets, just larger, so I set out to adapt what I had written and add in some client-side routing capabilities. This happened around August 2018.

I managed to do that. It wasn’t long before you could switch from the dashboard page to the new marketplace. The navigation on the side, now its own widget, automatically updated whenever you changed pages. Switching back to the dashboard was a little more complex, as the Angular application still existed and it would be a long time before the most complicated widget was converted to the Thin Controller framework.

Later on, the lifecycle of widgets/pages was updated to have a preloading step. This would automatically download the resources for another widget, but not actually inject them yet, under certain circumstances. This was used in our marketplace flow, which was a linear process across many pages. Using the preloading made the application feel faster, as the code for the next page would already be available.

Recursive widget injection

One minor challenge the marketplace brought us was an updating cart icon in the top navigation. As briefly mentioned, the navigation had become a widget at this point. The cart wasn’t really part of the navigation, so we made it into a separate widget. This just meant having the controller scan for more elements with data-widget-type after injecting the HTML.

This had the added side effect of being able to convert the dashboard page to the Thin Controller properly, since we could now place widgets inside another widget/page.

Polling and the API Cache

The issue of network request timeouts still plagued us, and with the Controller now handling almost all network requests, we saw an opportunity to save a lot of extra request time. The idea consisted of a few ideas working in tandem:

1. Add a way of telling the Traffic Manager of the Controller to poll for the final response using a token

2. Introduce a mechanism for storing response data in a temporary location in the database

3. If we’re storing responses in the database for a while, we may as well hold it a bit longer as a cache for those extra long requests

The change to the Traffic Manager to support this had no effect on its interface: widgets had no idea whether a request used polling or not. It involved a small async loop in the requesting function in case the server indicated that polling was required, doing polls with an exponential drop-off until the server responded with the final data.

The backend logic did get a little more complicated when the ability to refresh a widget and force it to bypass the cache. I still have a photo of the table I drew up on a whiteboard. To start with, all of the code to handle the database needed to be copy/pasted in each API’s handler. This grew too unwieldly, so I eventually wrote a wrapper for Java’s Callables to contain all that logic.

Polling was added to the Controller around May 2018, as I still have some photos of my planning on whiteboards from that time.

Notifications

A later requirement we got involved the addition of in-app notifications. There were some weird requirements around them as well, as there always are. I was heavily involved in both the front and backend implementations of the notifications.

I ended up implementing an event system into the Controller to better facilitate communication between widgets. Another use for this system was triggering updates for the cart icon in the navigation, but it turned out to work well for the notifications. I ended up creating a new widget for the notifications, and sending an event with the right fields would display a notification in the app. The notifications widget would also poll for notifications, and let other widgets know there’s a pending notification so the widgets could decide when to show them to the user. This meant there was some extra back-and-forth when implementing anything beyond a basic notification, which meant that other developers didn’t really know how the system worked.

The end of the Thin Controller

Simply put, the product we’d been working on got replaced by a rebuild made by a team internal to our client. Since it was a rewrite from scratch, they didn’t need the Controller, so once the new version took over, that was it for that project. We didn’t use the Controller in other projects because it just wasn’t necessary. It was a product of the unique circumstances we were in with that project; where there was no expertise in the team as to how to architect a Javascript application, but all we knew was cobbling things together to just work.

I don’t remember the Thin Controller for being an amazing world-changing work of engineering — it wasn’t. I remember it because it’s a symbol of the strange way my career got its start, and the equally strange circumstances I worked my way through.