These Frictionless Data Tool Fund posts are part of the grantee profile series, written to shine a light on Frictionless Data’s Tool Fund grantees, their work and to let our technical community know how they can get involved. Frictionless Data Tool Fund grantees are individuals and organizations that Open Knowledge International has commissioned to extend implementation of Frictionless Data libraries in additional programming languages.
I was born in Maceió, a sunny coastal city in the Northeast of Brazil. It was 20th century still when I had a first contact with an Intel 80386 and installed Conectiva Linux Guarani 3.0. A lot has happened since, for instance, a bachelor's degree in Computer Science at UFCG after three years as a research assistant in the Distributed Systems Lab (LSD). It was already the 21st century when I realized that distributed and scalable systems were the way to go. I kept on studying the field and pursued a MSc at UFMG. From there I joined Google and spent 6 happy years working at multiple offices (NYC, ZRH, BHZ). I’ve got the chance to work on a myriad of projects, ranging from social networks to Google's default Java HTTP/RPC server framework. Currently, I'm back to UFCG doing a Ph.D. in cloud computing performance. It is easy to find me at hackathons and other efforts to increase transparency of public data. I have also been busy working on projects like contratospublicos.info and Frictionless Data, using Go to improve data transparency in Brazil and around the world.
I started following Open Knowledge International (OKI) on Twitter after watching a talk from Victor Baptista at UFCG. I learnt about Frictionless Data from posts by OKI and liked the overall idea a lot. I have been a Golang enthusiast for a while now, but I hadn’t thought of applying to the fund until I had a quick chat with Nazareno Andrade that started with Golang and ended with: “what about the Frictionless Data Tool Fund?”
Go has a lot to deliver in terms of approximating simplicity of reading/writing, correctness, and performance. I believe bringing the experience and solid specifications of Frictionless Data to the Go ecosystem will not only make data description, validation and processing easier and faster, but also help to decrease the distance between data analysis/processing and production serving systems, resulting in simpler and more solid infrastructure.
In the coming weeks, I hope to use the Tool Fund grant I received to bring Go’s performance and concurrency capabilities to data processing and to have a set of tools distributed as standalone and multi-platform binaries which are very easy to download and install. I am currently working on my Ph.D. and one pitfall I have come across is the use of one environment/system to collect/generate data and another to process. I will be working to alleviate this issue in order to make it easier to process tabular data in Go.
From the developer's perspective, it is really great to use open source software. This is especially true when the community around the software fosters it's usage and welcome contributors. That ends up increasing the overall quality of the software, which benefits all users.
The source code will be hosted at Github’s tableschema-go and datapackage-go repositories. We are going to use issues to track development progress and next steps.