Best Practices to Become a Data Engineer

Data Science And Machine Learning Big Data Data Dashboard

Pandas Landing Page
1

Pandas

Pandas is an open source library providing high-performance, easy-to-use data structures and data analysis tools for the Python.
Pricing:
- Open Source
If you are making a transition to a career as a data engineer, then the manipulation of data and the cleaning of data are going to become extremely important. The first step in this journey may be to take a subset of data, and to work with Pandas (a Python package which is “Excel on steroids”) in order to really understand the data.

#Data Science And Machine Learning #Data Science Tools #Python Tools 198 social mentions
Jupyter Landing Page

2

Jupyter

Project Jupyter exists to develop open-source software, open-standards, and services for interactive computing across dozens of programming languages. Ready to get started? Try it in your browser Install the Notebook.

Python for Data Analysis - This book by Wes McKinney is a couple of years old, but it gives a really good walk through of NumPy and how to use it in an interactive Python environment called a Jupyter Notebook.

#Data Science And Machine Learning #Data Science Tools #Data Science Notebooks 205 social mentions
unittest Landing Page

3

unittest

Testing Frameworks

#Automated Testing #Testing #Online Services 60 social mentions
Colaboratory Landing Page
4

Colaboratory

Free Jupyter notebook environment in the cloud.
Pricing:
- Open Source
Google Colabratory - If you are looking for a free resource to run Python, NumPy, and TensorFlow, you may want to try Google CoLab. This site allows you to run code using GPUs that work well with machine learning operations.

#Development #Education & Reference #Education 208 social mentions
Visual Studio Code Landing Page
5

Visual Studio Code

Build and debug modern web and cloud applications, by Microsoft
Pricing:
- Open Source
Visual Studio Code: This is a lightweight integrated development environment. As a side note, colleagues of mine swear by VS Code for Python development, but I have only used VS code for React, JavaScript, and TypeScript development.

#Text Editors #IDE #Software Development 1017 social mentions
Google Cloud Dataflow Landing Page

6

Google Cloud Dataflow

Google Cloud Dataflow is a fully-managed cloud service and programming model for batch and streaming big data processing.

Apache Beam - Apache Beam is a scalable framework that allows you to implement batch and streaming data processing jobs. It is a framework that you can use in order to create a data pipeline on Google Cloud or on Amazon Web Services.

#Big Data #Data Dashboard #Data Management 14 social mentions
Apache Beam Landing Page
7

Apache Beam

Apache Beam provides an advanced unified programming model to implement batch and streaming data processing jobs.
Pricing:
- Open Source
Apache Beam - Apache Beam is a scalable framework that allows you to implement batch and streaming data processing jobs. It is a framework that you can use in order to create a data pipeline on Google Cloud or on Amazon Web Services.

#Big Data #Data Dashboard #Data Warehousing 14 social mentions