How GraphQL changed our team
There are lots of articles and blog posts about how to develop APIs with GraphQL, but what about the non-technical part? Is there anything special? I bet there is.
When I joined the company almost 3 years ago the web was simpler. jQuery, Backbone, Sass — sounds familiar, doesn’t it? Now it is yarn, npm, node, webpack, react and many-many other things that one has to know to be a good frontend developer. It is hardly possible nowadays to sit on 2 chairs at the same time as many full-stack developers did in the past.
Not only technologies changed since that time, but how APIs are built as well. While REST is still REST, nowadays we may enjoy building endpoints with an alternative — GraphQL. What should we do to use it and why?
From the beginning of time
I am working for Profile team at XING — the market leading business network in German-speaking countries. Try it if you have not done this yet.
We provide tools for our users to express and identify themselves across the platform. As many things on the web, that tools were also obsolete over time and required some fresh breath.
To go further and fully rebuild the product we had to touch not only the frontend part but the backend one as well. That is where our main enemy was hidden.
For the decade we have been using tradition REST-like APIs for web and mobile. Every platform sets its own requirements on how APIs should be built and how certain data should be exposed. Problems arise when you want to change the frontend and so, change the requirements for existing APIs. That is where versioning comes to the scene. And if for the web, switching from one endpoint to another is a matter of one deployment, for mobile it is a big challenge — to go through the path from releasing a new app to adopting it by the most of the users. That is when your maintenance costs may shoot.
That is why we had to find a solution, and it was GraphQL. It utilizes existing APIs and provides one unified interface for all the platforms to query the data. The GraphQL is not a replacement for REST though. It just hides all the bad things you know from REST (versioning, over / under fetching and such) and gives you a nifty interface for querying the data.
When we started our project in 2016 our team reflected the old cosy web. In other words, it was a web-centric team with all the consequences, like how APIs were developed and so on.
This setup caused some friction time to time between us and the mobile developers. GraphQL could be our saviour by providing one unified API endpoint. The only thing we had to care about was to find a way how to cooperate.
Despite GraphQL sounds like something similar to REST (it is dealing with API in the end), one must be prepared for certain things. Our first prototype was built within 8 hours and we were naïve thinking we could release it rather soon. Unfortunately, It took us literally 5 long months to cast it into something shippable.
GraphQL requires one to change his way of thinking how APIs should be built. That is how we realised that the journey we just started would be much longer than we had thought initially and definitely would not be so easy.
By looking at other market players, who already had been living with GraphQL for a while, we started carving our brains and the schema into the right direction. That resulted in enormous amount of discussions around how certain data should be exposed to the clients. But after all the wavering we set the process.
Every new wish, bugfix or idea now on must be presented as a GitHub issue. When discussions (if any) are over there and we find somehow a common ground, we switch to a pull request with actual changes to the schema, using GraphQL IDL (Interface Definition Language).
On that step, we do not expect much discussions as the most critical parts are supposed to be solved on a previous level. Inquisitive persons may use graphql-faker tool that will run a mock server with the changes and one can play around with it. Whenever the pull request is ready and approved by whoever is interested in it, a backend engineer needs just to implement the changes.
That procedure helped us to move forward. Not everything was perfect, not everything is even perfect now, as every sub-team (ios, android, web, designers, and such) has their own vision of how things should work.
Sometimes we see a resistance in accepting the GraphQL’s culture and its best practices. Especially, when it interferes with commonly used patterns on certain platforms. Or, when it requires the particular sub-team to refactor the legacy code before switching to GraphQL.
Yet a bit more things about GraphQL
As I mentioned before GraphQL is not REST. It may (but not necessary) have a direct connection to any database as it often happens with REST endpoints, but it also allows you to put it on top of existing APIs.
In any case, you, as a developer do not care about WHO will query it, but rather HOW and for WHAT purpose. Just like with any relational databases, where you care about indexes, tables, relations between them, but not about who will consume them.
But if to gather the data from, let’s say, MySQL, you use SQL, that you know from your university age, GraphQL in contrary operates with different things — types, nodes, connection, unions, interfaces and such.
Mainly, it is about types. This is a core element of GraphQL. Every node must have a type. It can be a simple (Scalar) one like String or Integer or a custom defined complex type with multiple subtypes inside. Please, check the official GraphQL website for more details.
Every node may have an associated function which resolves the node. In GraphQL terms, this is called “resolver”. It transforms the incoming data from your underlying REST endpoint, database or any other data source so that it can be mapped to the node structure.
All that requires you to think twice about how to represent so well-known data, that previously you could get from REST, into one big schema.
Ruby or not to Ruby? That is the Question
When we started the project, GraphQL was not a big part of it. It was more like one of the possible ways of solving some technical issues with APIs. That is why we picked graphql-ruby gem since all our apps are written in ruby and this was kind of a “safe ground” for us.
We were not alone here. The API team, whose project was to re-wamp API in general for the whole company, also struggled for a while with choosing the proper technology. GraphQL was also just an option, a possible way to go.
They started from scratch and had completely different constraints and requirements and in the end, they picked GraphQL and Sangria — Scala based implementation of the technology. Why? They compiled a huge list of pros and cons to prove the benefits of every available implementation, but to cut the story short, this is the most feature complete GraphQL-implementation in the world.
The joint operation
Having similar deadlines and some common intersections both teams, I mean, we and the API one did what we always do at XING to achieve the goals — we cooperate. Even though we both were busy with our own stuff, we were constantly discussing the schema.
We were carving this stone together by adding things from the perspective of each team. In cases where we were too profile-centric, the API team diluted the schema with company-wide concerns.
In May 2017, we released our internal “alpha” powered by our own ruby-based GraphQL instance. At the same time, the API team finished their prototype and so proved the whole concept. Our next step was to unite the projects.
Thanks to the time we both spent discussing the schema switching from one endpoint to another were almost painless for the clients (web, ios, android). It was not so difficult for backend team as well.
Now on we use Scala and special DSL designed by the API team to extend the schema. However, that is not completely true.
Types, resolvers and other GraphQL’s stuff that is written in Scala is just our present, or, to be more precise — is almost our past. The future is to modify the schema using IDL — the same domain-specific language that we used the whole 2017 to discuss and try out changes to the schema.
The GraphQL interface definition language (IDL), that is though not part of the GraphQL specification yet, it is the de facto standard to define schemas in various GraphQL implementations in a technology agnostic manner.
That means, that developers who want to extend the schema do not have to re-compile the whole project on every change but just commit the fixes in a simple text file, that will be magically applied on the server once being deployed. Amazing.
Almost at the same time
Not only the process of developing APIs was changed during the year. We also touched the way we release things. Traditionally, the release flow for native apps (no matter which platform we are talking about) has several steps that you cannot skip or speed up. At XING we call this flow a Release train which takes place several times a month for every native platform. Here we must keep in mind not only our own deadlines but the dates when we are able to deliver things. The Web team, however, has completely different workflow and is able to ship things almost every hour.
Switching from diverse set of APIs to one GrahpQL endpoint made it possible to ship changes to all the platforms almost at the same time. Instead of versioning or even releasing a new API for every consumer, we just extend the schema with new fields or deprecate old ones. Old clients just behave as usual, while new clients can pick what they want and ship changes when it is applicable to them.
To deliver things simultaneously, however, the backend team now has to start a little bit earlier to spend some time on API discussions, spikes and investigations. By the time the first frontend engineer picks the task the backend team already offers a signature of API or, sometimes, even a finished schema.
Moreover, even the life of our QAs was changed radically. Instead of testing all the layers separately for every platform, they now are able to test the whole stack at once.
All that, actually, brings me to the end of my story. Was our journey interesting? No! It was exciting. We went from a web-centralised team having legacy REST endpoints to a decentralised cross-platform one with a unified API, powered by a fancy GraphQL service, that is called X1 (XING One) by the way.
That made it possible for us to get rid of tensions, in some sense, that happened time to time between the sub-teams (QA, frontends, design, backend) and increase the performance. Should I even notice what happened to the overall quality? All that, of course, required us to think different © but it’s worth the effort in the end.