What would you do when you have over 2 billion data profiles on your systems? Let us say this much: There’s never really a lack of things to work on for our engineers, from ingesting and processing data to visualizations and finding smart ways to segment the data into actionable and useful data products. Our team’s approach to taking a break to find clarity and unlock smart hacks, data product enhancements and saving potentials was – well one could even say pretty “German” in nature: a 3-day hackathon.
A spirit of competition was in the air. After several weeks of preparation the time to kick-off our first “zeo-Ode-To-Code” Hackathon had arrived. 3 days, 37 attendees, 48 idea submissions and countless pizzas and coffees would await our teams. And while our Product, Engineering, Infrastructure and Data Science teams usually work in unison – this time would be different. You should know we are huge fans of cross-functional knowledge sharing and out-of-the-box thinking. So we sliced up our every day teams and threw the dice to build multi-functional teams.
Since early morning, the different meetings rooms, tea rooms and the rooftop terrace were occupied by zeotap team members huddling together with their laptops. Altogether, nine teams of four to five members. There were no boundaries for the idea generation, neither for the team names. And so “The Rippers” and “Bounty Hunters” started brainstorming about new verticals and products, focused on building something on top of our existing solutions or simply tried to make them better. Most teams finalised their ideas during the first day of the hackathon. Others took a more philosophical approach and were still debating until late night on what the most promising areas of focus would be and how they would shape up with zeotap’s company vision.
Day 2 showed the true nature of our colleagues: friendly but competitive. Well, in parts it was warfare. A friendship during pizza time could turn into distraction tactics and while the battles were ongoing on the rooftop, other teams were playing Switzerland and locked themselves up on a mountain of insights and took good care to not leak any hint about their solution or pitch.
From the 9 teams, a few ideas evolved and shaped that deserve a particular mention while many others are now pursued that were discarded due to a lack of time or clarity during the hackathon itself.
Project zScalar truly made it up to the stars as the name indicates. Spending a lot of money on server capacities, the team developed a UI in no time to optimise spot market bidding on Amazon Web Services to bring down our server costs significantly. The solution replaced a so far manual process and helped us buy server capacity in an instant upon high demand and scale it down when all jobs are completed and included an intelligent bid cancellation logic.
Another project, called “zeo Studio Lite” turned out to be an intuitive drag & drop web-based UI for data ingestion which would help us remove all the friction from another previously manual process that would bring down data ingestion times for new partners from a few days to a few hours.
Yet another team optimized our cloud computing expenses: it not only provides insights and recommendations about orphaned resources but also transparency about the resource utilization. With its complete automation, the idea was to enable the reduction of our cloud computing costs by 25%.
The all-female team “ladybugs” that had no bugs in their project output, was working on a project named “Zeo-Gest” which is short for zeotap Segment Suggest. A segment discovery platform for specific brands and campaigns, the tool would be able to suggest the right combination of targeting data from a large variety of attributes upon the simple entry of a client’s industry and campaign goals.
Then there was a Data Science project. 6 models were established to estimate user wealth, credit worthiness and annual income. To establish a legitimate model, the team trained a model using 600 million data points and predicted missing attributes for over 200 million mobile devices. They also crawled many publicly available datasets to increase the accuracy of the model and validate their assumptions.
Finally, a project named “Upflow” aimed at making our many channel partner integrations with player such as Google DBM, Adform, The Trade Desk more fluent. Taking several days, the existing process was already in good shape but the team was seeking the challenge to think beyond industry benchmarks. The result was a UI that is able to create workflows that reduce the intervention of our engineering teams to a bare minimum. The solution went live immediately after the hackathon and chai consumption has spiked up across the office and is used to contemplate further best practices.
The three-day event culminated in an idea pitch competition to a judging panel of our senior management. The judges had followed the teams’ process curiously during the last two days. Now it was about the teams’ ability to convey their ideas and solutions.
Panel from left to right: Aditya Chandra – Director of Infrastructure, Ameya Agnihotri – SVP Engineering & Technology, Swapnasarit Sahu – VP of Data Science, Nikunj Jhunjhunwala – VP of Product and Sathish KS – Director of Engineering.
Some of the factors that the panel had to take into consideration when determining the final verdict were things like business impact, business viability, the originality of the idea and its disruptive potential, the overall quality and completeness of the demo, the technical depth and finally the WOW factor of the pitch. With a trip to Thailand for the winning team the pitches were not only elaborate but also filled with funny skits about zeotap’s engineering culture and jokes around the product pain points.
“Things really started blazing and almost all ideas were production worthy. Overall it was super-successful!” Sathish KS – Director of Engineering
In the end, Chaitanya, Aman Mangal, Shantanu and Ashish were stepping on the winner’s podium for the zeotap hackathon. They were the ones developing “zScalar” which significantly improved zeotap’s server infrastructure costs thanks to an innovative algorithm. Within only 3 days they had outperformed the solutions of many third-party vendors and save zeotap the expensive licensing costs for their solutions! The team also surprised the jury with already having deployed this technology on live production jobs and could show that all features were working. The second prize, a generous voucher for different experiences and gifts of choice, went to the team that had developed “zeo Studio Lite” helping zeotap to become more customer-centric. And eventually, everyone received a prize of some sort.
By the time the winners were announced it was already 11 pm and three days without much sleep were wearing down even the most renowned night owls. A last tired selfie and the teams went home satisfied and inspired with ideas and having built new deeper bonds with colleagues across departments.
“It was a great way of making use of such a talented team and spanning team spirit across the different departments.” Ashish Goswami – Sr. Software Engineer
Ultimately, 7 out of the 9 projects were production worthy, while the 2 winning projects ended up being implemented immediately. Some outcomes were clearly worthy of founding spin-offs of zeotap. And that is what we call fuel for growth!