{ dist sys, formal verification, event streaming }
We previously created a tokenizer that breaks up a sequence of characters into a sequence of tokens (enum TokenType) and a LL2 production notation grammar that acts as a template for the code of the parser.
The input of this parser will be the sequence of tokens and the output will be an Intermediate Representation (IR) which is a data structure that represents the DSL text in a structured manner. The next step, after parsing, will be translating this IR into SQL.
When you use a parser generator like ANTLR, the grammar is fed in and the code of a parser is the result. However we are hand crafting a parser so for us the grammar will act as a reference and will ultimately map neatly onto our code.
There are different styles of grammars and we'll look at two different ways of expressing a grammar.
Before you read on, there is a more efficient version of this tokenizer here: http://jack-vanlightly.com/blog/2016/2/24/a-more-efficient-regex-tokenizer that has significantly better performance.
So a tokenizer or lexer takes a sequence of characters and output a sequence of tokens. Let's dive straight into an example to illustrate this.
Meet a simplified version of Logging Query Language (LQL)
Creating your own DSL is fun, it involves multiple complex steps which can be challenging and very rewarding to figure out. However if you don't know what you're doing then your code can end up one big hack which, while works, is complicated, hard to change and hard to read. Although pleased that you have a working DSL you know that underneath it's no looker. In this series I will go through, step by step, creating a simple query language that gets translated into SQL and executed against SQL Server.
Having worked in a centralised application support team I have had the opportunity to study the personality traits that make a good fit for support and which don’t. When I say support I am talking about supporting the operation of an application in production, whether it is in a centralised team or in a dev ops team that supports what it builds.
What is it about programming that most programmers love? Creativity, problem-solving, analysis and critical thinking are just a few. However, when a senior developer explains to a junior, to the letter, how to implement something, we are taking away those great things about programming from the junior and we are taking away the opportunity for learning. But that doesn’t mean that we should give free rein to anyone in the team to implement solutions as they wish. It all depends on the level of the developer and the complexity of the problem/solution space.
LQL came about because of a log analytics platform that I developed for Vueling Airlines. Being a backend developer and a self-confessed newbie at front end (HTML, CSS and Javascript), I originally created LQL in order to avoid having to develop complex front end code for generating queries. I decided that instead I would create a Domain Specific Language (DSL) that would allow users to write the queries and all I would need to provide was a text area.
As a team leader you are directly responsible for the moral and attitudes of your team. One of the most important things you can do is strive for an attitude of team positivity, comradery and togetherness. If you get this right then you are on track to having a successful team, get this wrong and no matter what else you do you’ll hit a wall and you won’t be able to progress. Not to mention that working in a team that likes each other and believes in what they do is really fun and rewarding.
Most developers start out their career with an aversion to knowing the failure modes of the software that they write. They treat the software that they write as their baby, and don't like to see it get hurt. This isn't necessarily a conscious decision but more a subconscious mind-set. So when I am coaching a junior, one of the things I concentrate on is breaking them out of this mind-set.
