Parsing Lambda Error Logs in ReScript & Python

1

OCaml

ReScript code is just like F# or OCAML; it doesn’t have a function parse phase like JavaScript, so we have to define our functions and types first before we can use them. That’s fine, but makes explaining the code backwards (meaning you start at the bottom of the file and work your way up), so we’ll start at our lambda handler and explain each part, regardless of where it’s defined.

#Programming Language #OOP #Generic Programming Language 30 social mentions

2

Node.js

Compile and run quickly, getting the confidence in your code, a fast feedback loop, leverage existing Node.js & JavaScript ecosystem for AWS.

#Runtime #JavaScript Runtime #JavaScript 789 social mentions

3

mypy

If you’re an imperative or Object Oriented programmer in Python, this may look not just unpythonic, but super weird. We write functional code in Python much like we’d do it in JavaScript or ReScript; by using a Monadic interface provided by either the language (e.g. Promise in JavaScript or Result in ReScript), or a library (e.g. Folktale for Result in JavaScript). In Python, we’re using PyMonad. While dry-python/returns is super legit and has great integration with mypy, we’re using raw Python here with no Typings. The hope is “something as simple as an Error log parser for a single Lambda is perfect for dynamic Python” ya? While we can capitulate on strong/sound types, we’re not renegaging on Functional Programming for it’s benefits.

#Code Coverage #Code Analysis #Code Quality 49 social mentions

4

Mochajs

To do that, we’re using rescript-test, which feels natural to those used to Jest, Mocha, etc. The only main difference is using testAsync vs test for async code, and having to define your own assertions vs. “Jest/Chai has ALL THE THINGS”.

#Development Tools #Javascript UI Libraries #JavaScript Framework 86 social mentions

5

Lo-Dash

Instead of writing the above, we can use a pure function lense function from PyDash (the Python version of JavaScript’s Lodash). It has a function called get which allows us to safely access not just properties, but deeply nested ones, including a mix and match of Dictionaries and Lists. Instead of the above style for a deeply nested property:.

#Development Tools #Javascript UI Libraries #JavaScript Framework 85 social mentions

6

GraphQL

If you’re building an API, whether a REST one using API Gateway, or a GraphQL one using AppSync, you’ll have a monorepo with a bunch of Lambda functions in it. Each Lambda function is 1 to many files, and each function typically corresponds to a REST API route, or a GraphQL query/mutation. You’ll use something like Serverless Framework, AWS SAM, or AWS CDK to deploy. The deployment uses CloudFormation behind the scenes which means when you first deploy, you’re deploying “everything”. If you change 1 line of code in 1 Lambda and deploy, you’re only deploying 1 Lambda function update. Using Serverless deploy function, or AWS SAM accelerate, you’re bypassing CloudFormation to quickly test code in a dev/qa/stage environment, something common in Serverless. You can, and should, run and test your code locally, although for some runtimes (Go, Python with C++ libraries, custom runtimes, etc) this can be difficult so you just deploy and test in AWS only.

#Developer Tools #Web Frameworks #JavaScript Framework 223 social mentions

7

F#

ReScript code is just like F# or OCAML; it doesn’t have a function parse phase like JavaScript, so we have to define our functions and types first before we can use them. That’s fine, but makes explaining the code backwards (meaning you start at the bottom of the file and work your way up), so we’ll start at our lambda handler and explain each part, regardless of where it’s defined.

#Programming Language #OOP #Generic Programming Language 19 social mentions

8

AWS Step Functions

Alarms, however, merely say things like “Your Lambda crashed 3 times in the past 5 minutes”. They don’t answer why it crashed. In Imperative or Object Oriented Programming, crashes are fine; they’re often intentional. In Functional Programming, this is typically not done; crashes, while not academically defined as side effects, in practice are bad, unintentional side effects. However, the good news is AWS Lambda’s contract is “when you’re Lambda doesn’t work, you’re supposed to throw an Exception to signal to AWS”. This is good, and how things like SQS retries work, how Step Functions interpret whether they should retry or exponentially backoff, etc. That’s good for operations, but not at all helpful for monitoring and support.

#Project Management #Workflow Automation #Web Service Automation 58 social mentions

9

Amazon SQS

Alarms, however, merely say things like “Your Lambda crashed 3 times in the past 5 minutes”. They don’t answer why it crashed. In Imperative or Object Oriented Programming, crashes are fine; they’re often intentional. In Functional Programming, this is typically not done; crashes, while not academically defined as side effects, in practice are bad, unintentional side effects. However, the good news is AWS Lambda’s contract is “when you’re Lambda doesn’t work, you’re supposed to throw an Exception to signal to AWS”. This is good, and how things like SQS retries work, how Step Functions interpret whether they should retry or exponentially backoff, etc. That’s good for operations, but not at all helpful for monitoring and support.

#Data Integration #Stream Processing #Web Service Automation 65 social mentions

10

AWS Lambda

When Lambdas log to CloudWatch, it’s a stream a messages. Those streams can be subscribed too. Those streams can also be filtered to “only give me the ones that have errors in them”. You can then have that filtered stream invoke a Lambda with log error message, parse it, and send the error somewhere. This allows you to build better reactive, and proactive, monitoring, like sending error messages to Slack/PagerDuty/xMatters. Instead of “something broke” you get “the database couldn’t be connected to” or “an upstream service had a blip of downtime, but all is fine now”. Much more useful and actionable and nice to get English on your phone when it’s 3am you’re out of it vs “some error, please acknowledge page”. “NO, I demand you tell me what is exactly wrong waking me up at an un-godly hour!” “pLe@$3 ACK-kN0wl3DgE p@ge” “Stupid phone……”.

#Cloud Computing #Cloud Hosting #Backend As A Service 248 social mentions

11

Amazon CloudWatch

When Lambdas log to CloudWatch, it’s a stream a messages. Those streams can be subscribed too. Those streams can also be filtered to “only give me the ones that have errors in them”. You can then have that filtered stream invoke a Lambda with log error message, parse it, and send the error somewhere. This allows you to build better reactive, and proactive, monitoring, like sending error messages to Slack/PagerDuty/xMatters. Instead of “something broke” you get “the database couldn’t be connected to” or “an upstream service had a blip of downtime, but all is fine now”. Much more useful and actionable and nice to get English on your phone when it’s 3am you’re out of it vs “some error, please acknowledge page”. “NO, I demand you tell me what is exactly wrong waking me up at an un-godly hour!” “pLe@$3 ACK-kN0wl3DgE p@ge” “Stupid phone……”.

#Monitoring Tools #Log Management #Application Performance Monitoring 54 social mentions

12

AWS CloudFormation

If you’re building an API, whether a REST one using API Gateway, or a GraphQL one using AppSync, you’ll have a monorepo with a bunch of Lambda functions in it. Each Lambda function is 1 to many files, and each function typically corresponds to a REST API route, or a GraphQL query/mutation. You’ll use something like Serverless Framework, AWS SAM, or AWS CDK to deploy. The deployment uses CloudFormation behind the scenes which means when you first deploy, you’re deploying “everything”. If you change 1 line of code in 1 Lambda and deploy, you’re only deploying 1 Lambda function update. Using Serverless deploy function, or AWS SAM accelerate, you’re bypassing CloudFormation to quickly test code in a dev/qa/stage environment, something common in Serverless. You can, and should, run and test your code locally, although for some runtimes (Go, Python with C++ libraries, custom runtimes, etc) this can be difficult so you just deploy and test in AWS only.

#DevOps Tools #Continuous Integration #Continuous Deployment 113 social mentions

13

AWS AppSync

If you’re building an API, whether a REST one using API Gateway, or a GraphQL one using AppSync, you’ll have a monorepo with a bunch of Lambda functions in it. Each Lambda function is 1 to many files, and each function typically corresponds to a REST API route, or a GraphQL query/mutation. You’ll use something like Serverless Framework, AWS SAM, or AWS CDK to deploy. The deployment uses CloudFormation behind the scenes which means when you first deploy, you’re deploying “everything”. If you change 1 line of code in 1 Lambda and deploy, you’re only deploying 1 Lambda function update. Using Serverless deploy function, or AWS SAM accelerate, you’re bypassing CloudFormation to quickly test code in a dev/qa/stage environment, something common in Serverless. You can, and should, run and test your code locally, although for some runtimes (Go, Python with C++ libraries, custom runtimes, etc) this can be difficult so you just deploy and test in AWS only.

#Project Management #Workflow Automation #Web Service Automation 30 social mentions

14

Amazon API Gateway

If you’re building an API, whether a REST one using API Gateway, or a GraphQL one using AppSync, you’ll have a monorepo with a bunch of Lambda functions in it. Each Lambda function is 1 to many files, and each function typically corresponds to a REST API route, or a GraphQL query/mutation. You’ll use something like Serverless Framework, AWS SAM, or AWS CDK to deploy. The deployment uses CloudFormation behind the scenes which means when you first deploy, you’re deploying “everything”. If you change 1 line of code in 1 Lambda and deploy, you’re only deploying 1 Lambda function update. Using Serverless deploy function, or AWS SAM accelerate, you’re bypassing CloudFormation to quickly test code in a dev/qa/stage environment, something common in Serverless. You can, and should, run and test your code locally, although for some runtimes (Go, Python with C++ libraries, custom runtimes, etc) this can be difficult so you just deploy and test in AWS only.

#API Tools #APIs #Web Service Automation 95 social mentions