View this email in your browser

Software Weekly - 07/29/2019

Data engineering landscape, serverless chaos, real-time image filtering, developer report, Android side-channel attacks & more


Last week’s most popular episode: Data Engineering with Tobias Macey. Tobias gives his perspective on the evolving landscape of data engineering, and the trends he is seeing on the Data Engineering Podcast.

Old favorite: Data Engineering with Pete Soderling. Pete discusses the rise of the data engineer, and how that role interacts with data scientists, software engineers, and machine learning experts. Pete runs Hakka Labs, a site that aggregates material for data science and data engineering. He also organizes DataEngConf, a conference for data engineering.

Cruise is a San Francisco-based company building a fully electric self-driving car service. There are opportunities for backend engineers, frontend developers, machine learning programmers, and many more positions. At Cruise you will be surrounded by talented, driven engineers--all while helping make cities safer and cleaner. Apply to work at Cruise by going to

Interesting Picks in Software Engineering and Architecture

Injecting Chaos to AWS Lambda functions with Lambda Layers. Check out how to run Chaos engineering experiments in Serverless functions. Some really great advice like: don't run this in production until you test in your staging environment first.

When a rewrite isn’t: rebuilding Slack on the desktop. This is a crazy rewrite story. The rules were old code cannot directly import new code and new code cannot directly import old code. This was a complete rewrite with old and new code written side by side. This required a lot of discipline I am sure.

Production Readiness. Amazing collections of guidelines and best practices for running in production.

Gitlab's 2019 Global Developer Report: DevSecOps. Some highlights:
  • Developers are 1.4x more likely to feel innovative if they have a mature rather than a poor DevOps maturity model.

  • Remote can bring you closer: Developers are 23% more likely to have good insight into what colleagues are working on when they have mostly remote teams.

  • All-remote teams are 1.6x more likely to quantify and document their work than in-office teams.

Presto at Pinterest. Presto is an open-source distributed SQL query engine. Pinterest has a very sophisticated and complicated Presto setup with layers. A fascinating read into the complexity of what might seem like a simple system.

(Can’t) Picture This 2: An Analysis of WeChat’s Realtime Image Filtering in Chats. A deeply technical report about how WeChat is doing the censoring in real-time using hash-indexes, using fingerprints for content to filter, while in the background they run computationally intensive OCR-based and visual-based algorithms.

Using OpenAPI to Build Smart APIs for Dumb Machines. OpenAPI 3.0 now allows for additional expressivity that can let machines do even more useful work for us. OpenAPI can drive powerful testing automation, it can be used to produce realistic mocks, and it can even generate native bindings that abstract away complexity from the developer.

How CircleCI Processes Over 30 Million Builds Per Month. An interesting look at CircleCI's organization, and their tech choices for frontend and backend .Turns out you need a lot of tech for a CI system.

Decoupling database migrations from server startup: why and how. The principle is good, but it takes significantly more work to do a migration that doesn't affect uptime.

Standby in production: scaling application in the second largest classified site in the world. The article explores problems and solutions in replication-based horizontal scale-out specifically for PostgreSQL, which the site runs on.

Design Systems (Part I: Foundations). If you've heard the term design systems so much in the last few months but have no idea what it means, here is your guide. There is something to learn for those working on frontend and backend.

Harnessing the Power of Neo4j for Overhauling Legacy Systems at Adobe. Adobe went from 125 MongoDB servers to 48 Cassandra servers to 3 Neo4j servers, Choosing the right database for your workload really makes all the difference. 

Brace Yourself for These 9 Database Horror Stories. Some will make you laugh. Some will make you cry.

With MongoDB Atlas, you can take advantage of MongoDB’s flexible document data model as a fully automated cloud service. MongoDB Atlas handles all the costly database operations and admin tasks that you’d rather not spend time on, like security, high availability, data recovery, monitoring, and elastic scaling. Try MongoDB Atlas for free today! Visit to learn more.

What's New in Tech

New Attack Lets Android Apps Capture Loudspeaker Data Without Any Permission. Speech on the loudspeaker affects the accelerometer which can be abused in Android devices to capture conversation data. The takeaway: we need more restrictive permissions models on Android since anything can be exploited.

Consent Matters: When Tech Takes Remote Control Without Your Permission. Who owns your computer? Can Apple patch your computer without your consent?

Microsoft Invests In and Partners with OpenAI to Support Us Building Beneficial AGI (artificial general intelligence). Is this going to be used for Cortana and Azure? Or something else?

Arrested development: Cops dump Amazon's facial-recognition API after struggling to make the thing work properly. 15 months of wrangling and Orlando couldn't even begin testing AI cloud tech for population surveillance.

Google will now pay up to $30,000 for reporting a Chrome bug. I am not saying everyone should be a bounty hunter but if you were considering it, the incentives just got better.


New SEDaily app for iOS and for Android. It includes all 1000 of our old episodes, as well as related links, greatest hits, and topics. You can comment on episodes and have discussions with other members of the community.
FindCollabs is a place to find collaborators and build projects. FindCollabs is the company I am building, and we are having an online hackathon with $2500 in prizes. If you are working on a project, or you are looking for other programmers to build a project or start a company with, check out FindCollabs.

Upcoming conferences Jeff is attending: Open Core Summit September 19th and 20th in San Francisco.

SE Daily is hiring two interns for software engineering and business development! If you are interested in either position, send an email with your resume to with “Internship” in the subject line.

GoCD is a continuous delivery tool from ThoughtWorks. If you have heard about continuous delivery, but you don't know what it looks like in action, try the GoCD test drive at GoCD's test drive will set up example pipelines for you to see how GoCD manages your continuous delivery workflows. 
Hi there, I'm Abdallah. I am a software developer myself and love talking news, insights, analysis of software engineering, and bringing you interesting content from across the web.
Your support for the show and the newsletter is greatly appreciated.

Thank you,
Abdallah Abu-Ghazaleh (@TheGhazStation)
Join the Slack community and discuss your thoughts with fellow listeners.

Newsletter Feedback

Got feedback? Reply to this email and let us know your thoughts.

If you enjoyed reading this newsletter, I hope you will forward it to your friends and coworkers and ask them to subscribe.
Copyright © 2019 Software Daily, All rights reserved.

Want to change how you receive these emails?
You can update your preferences or unsubscribe from this list

Email Marketing Powered by Mailchimp