Behind the scenes: Hacking our way to success

Jul 7 2014, 8:47 pm

Sreevatsan Raman is a software engineer at Continuuity where he is building and architecting a platform fueling the next generation of Big Data applications. Prior to Continuuity, Sree designed and implemented big data infrastructure at Klout, Nominum, and Yahoo!

We just wrapped up our latest hackathon and it was a great reminder of the unique engineering culture we have at Continuuity. We have created a new application development platform, Continuuity Reactor, which is focused on allowing developers to quickly and easily build Big Data applications.

Building a platform that no one has created before is a big challenge. We break this huge effort into a continuous cadence of platform releases that are delivered to production frequently. Before every release we take a break from our daily efforts and hack on our platform for 48 hours where we stretch our imaginations and the platform capabilities we just built.

Every hackathon gives us an opportunity to dog-food our technology. We come together wearing our developer hats to build features and applications, incorporating our lessons learned into continually improving the developer experience, with the goal of making Hadoop more simple and accessible.

One of my favorite aspects of our hackathons is how the whole company comes together to build cool stuff and have fun. From our CEO to our engineering team to people in non-technical roles, everyone participates. Here are some thoughts and experiences about our company, culture, and hackathons from our awesome engineering interns:

Shu Das, University of Michigan

The unique aspect of Continuuity that I like is that everyone has a clear sense of his or her agenda and responsibilities, so we’re empowered to stay on top of our game. Not only do I have the resources I need and responsiveness from the rest of the team, but also the working environment at Continuuity is lively and enjoyable.

My first project was building an application on Reactor that visualizes data about the test cases we run on our code. This work gave me great insights into what our platform is, how to use it, and how our technology can be used for simplifying Hadoop. I really appreciate the fact that the feature I worked on is used daily, as a component of the development lifecycle, and not left off as a side project.

For the hackathon, I teamed up with Kenneth and Gourav (see below) to build a Reactor application that can be used to aggregate, correlate, and visualize data - for instance, metrics, logs, or any other events. It was amazing to see the application built in a very short amount of time using new core functionalities of the platform and dogfooding the new APIs, runtime, and documentation.

Gourav Khaneja, University of Illinois

The work here is interesting because the problems we’re solving are hard. One of my favorite aspects of Continuuity is the willingness of team members to help each other to work through challenges. For example, even during crunch time, every Continuuity member is willing to stop what he or she is doing to help out a fellow employee. I learn a lot from the team on a daily basis.

When I joined, I was tasked with optimizing resource allocation in YARN using Apache Twill. YARN has a large codebase and although my previous experience with a large code base was limited, I was able to come up to speed quickly with great mentorship from the team and contributed towards a major feature in Twill.

Kenneth Le, University of California, Berkeley

Interns are involved in relevant projects right away. While we receive guidance when needed, the focus of the internship program is more on empowering us to deliver and learning more via open communication about the various projects that other people are working on.

My first project was improving a developer tool that is used to deploy code to clusters. The existing tool took about 30 minutes to build and deploy the entire code base. The newer version, which I rewrote in Python, takes about 6 minutes, thus saving developers a lot of time in their development life-cycle.

Julien Guery, Ecole nationale supérieure des Télécommunications de Bretagne

This is an extremely technical company solving challenging problems. One of the first things I noticed is that the interns get to be part of the core engineering team and are involved in all aspects of the company.

In my first project I learned a lot about Apache Hive and the Reactor platform while working on a feature to bring ad-hoc quering capabilities in to our platform. I had great mentors who taught me how to test and debug and gave me insights into the architecture of the systems, and now I can dive right into new projects and teams without fear.

During the hackathon, I used our APIs to build a Python SDK. I wanted to showcase how Python developers can easily write big-data applications using our platform and my efforts during the hackathon demonstrated how this could be accomplished. The hack was well received and a updated version of this SDK will be made available in a future release.


Our team is working to solve a difficult problem – making Hadoop a platform upon which data applications can be built by all developers. Whether at our hackathons or at our weekly company-wide demos, we are constantly sharing and collaborating so everyone can understand the impact that they have and the context of how their contributions map to the overall vision and mission of the company.

If you’re interested in learning more about our culture and careers opportunities at Continuuity, check out http://continuuity.com/careers.

Comments
blog comments powered by Disqus