All Posts

Developers started to notice just how big our JavaScript package was and yeah, we knew. We weren’t ignoring the issues; after all, we don’t want the Sentry package to be the cause of a slowdown. But to reduce our JavaScript SDK package size effectively we had to account for shipping new capabilities, like being able to manage the health of a release and performance monitoring, while maintaining a manageable bundle size. After all, new features == bigger package - usually.

Event ingestion is one of the most mission-critical components at Sentry, so it’s only natural that we constantly strive to improve its scalability and efficiency. In this blog post, we want to share our journey of designing and building a distributed ingestion infrastructure—Sentry Points of Presence— that handles billions of events per day and helps thousands of organizations see what actually matters and solve critical issues quickly.

SDKs naturally increase in size over time. After all, it does take more bytes to implement more features. This is not a big deal for most languages—the relative size of each new feature is small, and load times and storage aren’t big concerns for code running on a server. Larger JS bundles mean longer load times, which in turn increase user misery, which then can cause the user to leave pages entirely.

Most plugin based models load all assemblies into a single shared context. This is a common approach because it has better memory usage and startup performance. The history and rules of assembly loading in .NET is convoluted; its current status makes it difficult (and sometimes impossible) to load multiple different versions of the same assembly into a shared context. Instead of trying to struggle with existing options we decided to build a new tool: Alias.

In today’s modern web stack it’s anything but. Full stack developers are expected to write JavaScript executing in the browser, interop with multiple database technologies, and deploy server side code on different server architectures (e.g. serverless). Without the right tools, understanding how a user interaction in the browser cascades into a 500 server error deep in your server stack is nigh-impossible. Enter: distributed tracing.

Recently, Sentry converted 100% of its frontend React codebase from JavaScript to TypeScript. This year-long effort spanned over a dozen members of the engineering team, 1,100 files, and 95,000 lines of code. In this blog post, we share our process, techniques, challenges, and ultimately, what we learned along this journey.

At its core, Sentry is a tool that alerts you to defects in your production software. But it does more than blast stack traces into your inbox: Sentry provides powerful workflows to help your team determine root cause, triage issues to your team, and keep tabs on ongoing concerns with comments and notifications. At the end of 2019, the Growth team made it our mission to make it easier for our users to invite their teammates to join them on Sentry. Our theory: improving the user experience of inviting users, as well as democratizing the process to include all team members would lead to a significant increase in team-wide adoption. (Narrator: it did.)

Sentry’s growth led to increased write and read load on our databases, and, even after countless rounds of query and index optimizations, we felt that our databases were always a hair’s breadth from the next performance tipping point or query planner meltdown. Increased write load also led to increased storage requirements (if you’re doing more writes, you’re going to need more places to put them), and we were running what felt like an inordinate number of servers with a lot of disks for the data they were responsible for storing. Here’s a look at how we attempted to understand which database system was right for us and how we adapted our approach when we encountered some unexpected challenges.

Other than Python, JavaScript is the oldest platform that Sentry properly supports, which makes sense considering many Python services (including Sentry itself) have a JavaScript front-end. The system that almost everybody uses to debug transpiled code (and the hopefully apparent subject of this blog post) is source maps. Today, we want to focus on some of the their shortcomings and why source maps cause problems for platforms like Sentry.

Over two years ago, Sentry started supporting its first native platform: iOS. Since then, we’ve added support for many other platforms via minidumps and recently introduced our own SDK for native applications to make capturing all that precious information more accessible. Now, the time has come to lift the curtain and show you how we handle native crashes in Sentry. Join us on a multi-year journey from our first baby-steps at native crash analysis to Symbolicator, the reusable open-source service that we’ve built to make native crash reporting easier than ever.