Dreaming and Breaking Molds – Establishing Best Practices with Scott Haines

1

Apache Storm

So Yahoo bought that. I think it was 2013 or 2014. Timelines are hard. But I wanted to go join the Games team and start things back up. But that was also my first kind of experience in actually building recommendation engines or working with lots of data. And I think for me, like that was, I guess...at the time, we were using something called Apache Storm. We had Hadoop, which had been around for a while. And it was like one of the biggest user groups was out of the Yahoo campus. It was called the HUG group, like the Hadoop Users Group. So they met for basically pizza and stories on Wednesdays once a month, which was really fun.

#Big Data #Data Management #Databases 11 social mentions

2

Apache Spark

For example, when I was at Yahoo, we did a lot of things where we had the ability to basically process data in stream. But we didn't have repeatable libraries we could easily use. So we had to invent everything. So it was like, oh, we want to create a session. So somebody starts a user journey, where do they go within a journey? And is it all within a 15 to 30-minute timeout from the last event? How do we understand how people are using something or interacting with it? And those types of things are a lot more difficult than when we're like oh, we could do it like X, Y, or Z. And that stuff was just for free when we started using Spark.

#Databases #Big Data #Big Data Analytics 56 social mentions

3

Solitaired

And then you start to put things together when you think about who likes action-adventure games. It's going to be somebody who's usually younger or usually older. But it kind of skips like the 35 to 50 age range just because people are usually busy. You don't have time for certain things. You have time for a game of Solitaire if you're feeling stressed out or something else like that.

#Card Games #Strategy Games #Online Games 4 social mentions

4

Slack

Scott: I really enjoyed just the communication space and being able to give people a chance to be heard. And that was, I think, one of the most fulfilling jobs that I had. And that was a place called Convo Relay, and it was a startup in San Ramon. It was deaf-owned and operated. And I was the person who I joined, and I'm like, all right, this is going to be a reverse challenge. The team was awesome, the brightest people I've ever worked with. The only difference was that we had the synthetic filter of I can't keep up with their sign language. And we're chatting through messaging. Back before even the pandemic, we used Slack and things. But it was a really fun challenge. And I enjoyed my time there in the communication space.

#Communication #Group Chat & Notifications #Chat 208 social mentions

5

NewRelic

Jonan Scheffler: Hello and welcome to Polyglot, proudly brought to you by New Relic's developer relations team, The Relicans. Polyglot is about software design. It's about looking beyond languages to the patterns and methods that we as developers use to do our best work. You can join us every week to hear from developers who have stories to share about what has worked for them and may have some opinions about how best to write quality software. We may not always agree, but we are certainly going to have fun, and we will always do our best to level up together. You can find the show notes for this episode and all of The Relicans podcasts on developer.newrelic.com/podcasts. Thank you so much for joining us. Enjoy the show.

#Application Performance Monitoring #Performance Monitoring #Monitoring Tools 80 social mentions

6

Apache Kafka

Scott: But yeah, so games is the first time where we started adding streaming data. So we were using Kafka, and it was like, oh, sweet. This is really fun and difficult to do. So it was like a fun, new challenge. How do we make sure that we didn't drop data? It's like, well, you'll never know. It's like, okay, what are the things that people use nowadays or do nowadays? It's like, well, you need to validate your data. You need to do all these things to understand how things are flowing through your system.

#Stream Processing #Data Integration #ETL 120 social mentions

7

Hadoop

So Yahoo bought that. I think it was 2013 or 2014. Timelines are hard. But I wanted to go join the Games team and start things back up. But that was also my first kind of experience in actually building recommendation engines or working with lots of data. And I think for me, like that was, I guess...at the time, we were using something called Apache Storm. We had Hadoop, which had been around for a while. And it was like one of the biggest user groups was out of the Yahoo campus. It was called the HUG group, like the Hadoop Users Group. So they met for basically pizza and stories on Wednesdays once a month, which was really fun.

#Databases #NoSQL Databases #Big Data 15 social mentions

8

Flipboard

Scott: Yeah. So there's more history behind it. I think the data bug itself one of the first projects I started on was something called Livestand back in the day when the iPad 1 was out. And it's like, everyone's moving to non-printed media. When I joined the Yahoo Livestand team, it was kind of a Flipboard app. And the sad thing is that it wasn't Flipboard, and Flipboard literally came out a year later. And it's like, why didn't we have a really cool effect?

#RSS Reader #RSS #Insight Management 4 social mentions

9

Databricks

Scott: Yeah. So this is a program that just kicked off, and it's for people not like working for Databricks who are part of the community and helping to teach and evangelize or mentor new people.

#Data Science #Data Dashboard #Database Tools 17 social mentions

10

Apache Avro

Scott: It's like a very large row of Avro data that had everything you could possibly ever need. It was like 115 columns. Most things were null, and it became every data type you'd ever want. It's like, is it mobile? Look for mobile_. It's like, this is really crappy. I didn't know about, I guess, the hardships of data engineering at that point. Because this was the first time where I was like, okay, you're on the ground basically pulling data now, and now we're going to do stuff with it. We're going to power our whole entire application with it. And I remember that just being exciting. The gears were turning. I was waking up super early. I wanted to go in to just to work on it more. It was the first thing where it's like, man, that's just like the coolest thing in the whole entire world.

#Development #OS & Utilities #Tool 12 social mentions