How we completed a huge refactoring of a software used by thousands of developers without breaking things.

How to Refactor and Not Break Things

Cross-region replication is a foundational subsystem in multi-region Sentry. This post explores our design process.

Designing Sentry's cross-region replication

How to transform unreadable CSS selectors to React component names.

Improving DX: From Unreadable CSS Selectors to Clear React Component Names

Before splitting our application database and infrastructure up, we wanted to have confidence in where the boundaries would be.

Removing risk from our multi-region design with simulations

This is how we manage to comment Sentry issues related to code you're modifying in a pull request within seconds.

How open PR comments work

We're making updates to how Performance Scores are calculated in the Web Vitals module, which will bring them closer to what your users experience.

How we improved Performance Score accuracy

How ASP.NET Core application developers can make the transition from JIT (Just-in-Time) to AOT (Ahead-of-Time) compilation, using the Sentry SDK for .NET as a case study.

Should you, could you AOT?

A comprehensive guide on how to publish binaries on npm without getting fired.

How to publish binaries on npm

CommonJS and ES modules are 2 sides of a coin. Node.js supports both of them. So, how can we improve the performance of Node.js loaders?

Improving Node.js loader performance

Almost 2 years ago, Sentry embarked on a project to bring true EU data residency to Sentry's customers. We decided to do it the hard way.

A $3,000,000 Dropdown

We repurposed a hashtable to make ClickHouse significantly faster for analytical queries

How Sentry queries unstructured data in ClickHouse 62x faster

How we rebuilt Codecov's code renderer from the ground up to be faster and more efficient, utilizing virtualization.

Better Code Rendering Through Virtualization

Learn how we built the autoinstrumentation in the Unity SDK via IL Weaving

Enabling Out-of-the-Box Performance Insights in Unity Games with the Sentry SDK

Building a React component that automatically updates its font size to fill its parent element as fully as possible.

Perfectly Fitting Text to Container in React

See what's happening in your app before your first line of code can even run.

Mobile App Launch Profiling

As we scale Sentry, we need to expand Postgres capacity by splitting up workloads across multiple primaries.

Splitting production databases with minimal downtime

Preact or Svelte, which framework is best for building an embedded user feedback widget?

Preact or Svelte? An Embedded Widget Use Case

Let's dive into mutation testing. In this post we talk about how mutation testing works, the state of it in JavaScript and we share our results of applying it to our JavaScript SDK repository

Mutation-testing our JavaScript SDKs

How we resolved incorrect Codecov bundle size reporting when using GitHub Actions.

How we fixed incorrect Codecov bundle size reporting

This post will outline learnings the Sentry SDK team had from releasing v8 of the JavaScript SDKs.

Sentry JavaScript SDK v8 - A Retrospective

We just surpassed over 7 million check-ins per day. Let's talk about what it took to go from prototype to production ready cron monitoring.

Scaling Cron Monitoring

An in-depth analysis of how we managed to cut the Session Replay SDK bundle size by 35%.

How We Reduced Replay SDK Bundle Size by 35%

Finding a needle in a haystack is really difficult. Why make it harder than it needs to be?

Lost in the Haystack: Optimizing an Expensive ClickHouse Query

The best way to figure out how overhead impacts you is to measure it yourself. Follow along as we show you how we went about measuring overhead on Sentry and how you can measure it on your own applications.

Measuring Session Replay Overhead

We recently migrated JavaScript/SourceMap processing to Rust where we were hitting a lock contention problem in our processing infrastructure that kept people up for a few days. What happened, why and how did we solve it?

A locking war story

Knowing the user interactions which happened in your app right before it crashed is crucial context information for fixing errors. Tracking interactions like click and swipes manually can be tedious, so we at sentry looked into ways on how to do that automatically for your Jetpack Compose enabled Android app. Learn how you can intercept any touch event, how to determine Composable identifiers and ultimately how our sentry Android SDK ties it all together.

How we built user interaction tracking for Jetpack Compose

What is a self identifying file, what are debug IDs and why do we want a fundamental change in the web ecosystem for source maps.

Self Identifying JavaScript Source Maps: The Case for Debug IDs

Sentry is a very fast-moving company. In just one month we merged 165 pull requests from 19 authors and changed over 800 files, with a total of over 22,000 additions and almost 10,000 deletions. By updating to Lerna 6 with Nx caching, we were able to reduce our CI run times by about 35%. 

How we reduced CI time by 35% with Nx Caching

With over 700 pull requests a month, making sure Sentry's test suite runs quicky is extremely important. This post will talk about our journey converting our component tests from Enzyme to React Testing Library that took almost 18 months, 17 engineers and almost 5000 tests.

Sentry’s Frontend Tests: Migrating from Enzyme to React Testing Library

Jetpack Compose, a new declarative UI toolkit by Google made for building native Android apps, is rapidly gaining traction. The main advantage of using Jetpack Compose is that it allows you to write UI code that is more concise and easier to understand. This leads to improved maintainability and reduced development time. The main advantage of using Jetpack Compose is that it allows you to write UI code that is more concise and easier to understand. This leads to improved maintainability and reduced development time.

Getting Started with Jetpack Compose

The mobile development ecosystem has always been very diverse, arguably more diverse than the web development ecosystem. Both React Native and Flutter have a declarative approach from the start, but with Android and iOS now joining the declarative bandwagon, we can see that the future of mobile development is declarative.

Mobile: The Future is Declarative

Sentry helps every developer diagnose, fix, and optimize the performance of their code, and we need to deliver high quality stack traces in order to do so. In this blog post, we want to explain why source maps are insufficient for solving this problem, the challenges we faced, and how we eventually pulled it off by parsing JavaScript.

How We Made JavaScript Stack Traces Awesome

One of Sentries core company values is “for every developer”. We want to support every developer out there with our tools. But not every developer uses the newest or widely adopted tech stack, so we also try to support older versions of libraries and frameworks. To make sure that our SDK works correctly we have around 450 automated tests in our test suite that run for each change we make to the SDK.

How we run our Python tests in hundreds of environments really fast

Profilers measure the performance of a program at runtime by adding instrumentation to collect information about the frequency and duration of function calls. They are crucial tools for understanding the real-world performance characteristics of code and are often the first step in optimizing a program. In this post, we’ll walk through how we built Sentry’s iOS profiler, which is capable of collecting high quality profiling data from real user devices in production with minimal overhead.

Building a Performant iOS Profiler

Relay is an open source project by Sentry that receives data from Sentry SDKs for pre-processing. We wanted to make Relay behave like an AWS Lambda Extension moving our service closer to your code, and decreasing the latency of your Lambda functions. In this blog post, I’ll share how we used the actor model to set up Relay to behave like a Lambda Extension.

Relay as a Lambda Extension Using the Actor Model in Rust

In the modern web, the JavaScript you write is often down-compiled using a compiler like Babel to make sure your JavaScript is compatible with older browsers or environments. In addition, if you are using TypeScript (like the Sentry SDK’s do) or something similar, you’ll have to transpile your TypeScript to JavaScript. Understanding how your code is being transpiled and downcompiled is important, because your bundle size is affected by your final generated JavaScript. This post is all about the technical prep work needed to ship a 0 bug reported major issue.

Understanding the Performance Impact of Generated JavaScript

Developers started to notice just how big our JavaScript package was and yeah, we knew. We weren’t ignoring the issues; after all, we don’t want the Sentry package to be the cause of a slowdown. But to reduce our JavaScript SDK package size effectively we had to account for shipping new capabilities, like being able to manage the health of a release and performance monitoring, while maintaining a manageable bundle size. After all, new features == bigger package - usually.

JavaScript SDK “Package Size is Massive” - So we reduced it by 29%

Event ingestion is one of the most mission-critical components at Sentry, so it’s only natural that we constantly strive to improve its scalability and efficiency. In this blog post, we want to share our journey of designing and building a distributed ingestion infrastructure—Sentry Points of Presence— that handles billions of events per day and helps thousands of organizations see what actually matters and solve critical issues quickly.

Sentry Points of Presence: How We Built a Distributed Ingestion Infrastructure

SDKs naturally increase in size over time. After all, it does take more bytes to implement more features. This is not a big deal for most languages—the relative size of each new feature is small, and load times and storage aren’t big concerns for code running on a server. Larger JS bundles mean longer load times, which in turn increase user misery, which then can cause the user to leave pages entirely.

How we trimmed the Sentry JavaScript SDK file size by 20%

Most plugin based models load all assemblies into a single shared context. This is a common approach because it has better memory usage and startup performance. The history and rules of assembly loading in .NET is convoluted; its current status makes it difficult (and sometimes impossible) to load multiple different versions of the same assembly into a shared context. Instead of trying to struggle with existing options we decided to build a new tool: Alias.

Alias: An approach to .NET Assembly Conflict Resolution

In today’s modern web stack it’s anything but. Full stack developers are expected to write JavaScript executing in the browser, interop with multiple database technologies, and deploy server side code on different server architectures (e.g. serverless). Without the right tools, understanding how a user interaction in the browser cascades into a 500 server error deep in your server stack is nigh-impossible. Enter: distributed tracing.

Distributed Tracing 101 for Full Stack Developers

Recently, Sentry converted 100% of its frontend React codebase from JavaScript to TypeScript. This year-long effort spanned over a dozen members of the engineering team, 1,100 files, and 95,000 lines of code.
In this blog post, we share our process, techniques, challenges, and ultimately, what we learned along this journey.

Slow and Steady: Converting Sentry’s Entire Frontend to TypeScript

At its core, Sentry is a tool that alerts you to defects in your production software. But it does more than blast stack traces into your inbox: Sentry provides powerful workflows to help your team determine root cause, triage issues to your team, and keep tabs on ongoing concerns with comments and notifications. At the end of 2019, the Growth team made it our mission to make it easier for our users to invite their teammates to join them on Sentry. Our theory: improving the user experience of inviting users, as well as democratizing the process to include all team members would lead to a significant increase in team-wide adoption. (Narrator: it did.)

How we grew Sentry's monthly active users by rethinking invitations

Sentry’s growth led to increased write and read load on our databases, and, even after countless rounds of query and index optimizations, we felt that our databases were always a hair’s breadth from the next performance tipping point or query planner meltdown. Increased write load also led to increased storage requirements (if you’re doing more writes, you’re going to need more places to put them), and we were running what felt like an inordinate number of servers with a lot of disks for the data they were responsible for storing. Here’s a look at how we attempted to understand which database system was right for us and how we adapted our approach when we encountered some unexpected challenges.

How to Mutate Data in a System Designed for Immutable Data

Other than Python, JavaScript is the oldest platform that Sentry properly supports, which makes sense considering many Python services (including Sentry itself) have a JavaScript front-end. The system that almost everybody uses to debug transpiled code (and the hopefully apparent subject of this blog post) is source maps. Today, we want to focus on some of the their shortcomings and why source maps cause problems for platforms like Sentry.

Building Sentry: Source maps and their problems

Over two years ago, Sentry started supporting its first native platform: iOS. Since then, we’ve added support for many other platforms via minidumps and recently introduced our own SDK for native applications to make capturing all that precious information more accessible. Now, the time has come to lift the curtain and show you how we handle native crashes in Sentry. Join us on a multi-year journey from our first baby-steps at native crash analysis to Symbolicator, the reusable open-source service that we’ve built to make native crash reporting easier than ever.

Building Sentry: Symbolicator

For most of 2018, we worked on an overhaul of our underlying event storage system. We’d like to introduce you to the result of this work — Snuba, the primary storage and query service for event data that powers Sentry in production.

All Posts