Skip to content

Writing Good Bug Reports

Jonathan McPherson edited this page Nov 20, 2018 · 3 revisions

Welcome! This is a guide for effective community bug reporting for the RStudio IDE; its goal is to help you write better bug reports.

Basics

A good bug report follows the Show, Don't Tell principle. It has three basic components:

  1. What you did.
  2. What happened.
  3. What you expected to happen.

The impatient reader can stop now, as a well-specified report featuring these three components alone will put you in the upper echelon of bug reporters!

Where is the bug?

When you're looking at a problem in RStudio, there are a number of places that it can be coming from. Just to list a few:

  • Your own R code
  • Code in an R package you're using
  • R itself
  • The RStudio IDE

For this reason, we get lots of bug reports for the RStudio IDE which turn out not to have anything to do with the IDE itself. We help here where we can, but many of these reports don't go anywhere. To try to avoid filing them, the most helpful question you can ask yourself is:

Does it happen in base R?

A good test to see if your bug is in RStudio or not is to run your R code outside RStudio, and see if you still see the same problem. In fact, if your bug has the basic shape required to try this -- I run this R code and observe this error -- it is probably not an RStudio bug.

To run your R code outside RStudio, use the RGui program, which comes with R, or use R in your operating system's terminal.

If you have the same problem in RGui that you did in RStudio, you're not looking at an RStudio bug; more than likely there's a problem with your R code, and possibly with an R package you're using. Getting Help with R is a good resource, as is the RStudio community forum. Use reprex to post a reproducible example!

If your R code works perfectly fine outside RStudio, make sure to note that in your bug report. You should also include the results of Sys.getenv() inside and outside RStudio -- environment variables are the number one cause of IDE behavior differences.

Is it a known issue?

Search the RStudio Issues List before filing a new issue. If you find that your issue has already been filed, don't walk away! Instead:

  • Upvote the issue (add a +1 reaction on Github) to let us know you're seeing it too, and to let the original poster know that they aren't alone.
  • If you have time, add a comment to the issue describing any details that aren't already present in the conversation.

What makes the bug occur?

Don't file a Shrug Report! The most important thing you can do is share a set of specific, detailed steps to make the bug happen. These steps should be:

Minimal

Try to eliminate any steps that aren't really necessary to make the bug happen. Most bugs don't need a long wind-up and can be triggered with a short sequence of actions, and there's a certain kind of fun in trying to find the most elegant way to reproduce them.

Self-contained

If at all possible, step 1 should be something like "start a new RStudio project" or "create a new file". You want the development team to be able to reproduce your bug reliably using nothing but resources embedded or linked in the bug itself. In addition to making it more likely that they will be able to reproduce the bug, it also makes it easy for them to verify that it's fixed!

If your bug does only repro with certain data or files, try to either:

  • trim the data or files until they are as small as possible while still demonstrating the problem, or
  • create synthetic data or files which demonstrates the problem instead.

Reproducible

Finally, you want your steps to make the bug happen every time. If it only happens sometimes, it's hard for us to know whether we've fixed the bug or whether we've just gotten lucky! Some bugs are truly nondeterministic, and arise from timing and environment issues outside your control. However, the vast majority of sometimes bugs are really bugs that happen every time once you have figured out the preconditions.

Sometimes I see this error when I knit R Markdown documents: ...

... Are there any documents that always make the error occur? If there are, can you try trimming bits off the document until you isolate the part that causes the error? If there aren't, can you tell us the 2-3 things you're doing before knitting?

When I load this huge, private dataset, the UI doesn't look right

...Can you share a small, public dataset that has the same problem? If not, can you share a smaller, sanitized portion of your private dataset in the bug?

... You get the idea.

We realize that a minimal, self-contained, reproducible example is an ideal which not every bug report can reach, but we promise that if it's obvious you've tried your best to make one, your bug will get a lot more traction.

What does the bug look like?

You'd be surprised how many folks take the time to describe what they're doing, but fail to describe the actual issue they encounter. Avoid saying that something "doesn't work". Instead, show, don't tell exactly what happened.

  • Did nothing happen when something should have?
  • Did the wrong thing happen? If so, what?
  • Did you see an error message? If so, what did it say?

When in doubt, share a screenshot showing exactly what the bug looks like when it happens. There are often surprising clues elsewhere in the IDE!

What did you expect to happen?

Sometimes a bug report is a feature request in disguise, and what seems like a problem is actually just something working as designed rather than as expected. For this reason, it's very helpful to tell us why you think the behavior you saw is wrong, if it isn't blindingly obvious.

Don't be afraid or embarrassed to file an issue that might be "as designed". If the IDE isn't working the way most people expect, that's a kind of bug, and we want to know about it!

That said, avoid disguising feature requests as bug reports; read our guide to Writing Good Feature Requests instead.

What problem are you trying to solve?

It's often helpful to describe the larger task you were trying to accomplish. Sometimes what looks like a bug is actually just a symptom of trying to put a round peg in a square hole. If you describe your larger task, we can often suggest a better way to do it, and suggest a workaround that can get you back in business before the bug is fixed.

Does the issue happen in the preview release or daily builds?

Even if you're using the latest stable version of RStudio, your issue might already be fixed in a newer build! If you're able, it's very helpful for you to try to reproduce your problem in the very latest builds. You can get them here:

RStudio Preview

RStudio Dailies

If you tried more than one version of RStudio, let us know every version you tried (for example: "I use RStudio 1.1.463, but I also tried the daily build, 1.2.9234, and saw the problem there too.").

And if your bug reproduces in the latest stable version but does not reproduce in a newer build, it's probably not worth filing. Unless the issue is urgent enough to be backported (this is rare), it will probably be closed as "already fixed".

What does your environment look like?

A minority of bugs only reproduce on certain systems. To help us make sure our environment matches yours when we try to reproduce, it's always helpful to know your:

  1. RStudio Version. Available in Help | About, or run RStudio.Version().
  2. RStudio Edition. Desktop? Server? Note that if you're using Desktop Pro or Server Pro, you're entitled to professional support, and can email support@rstudio.com with problems.
  3. Operating System Version. For example, "MacOS 10.14".

It is not generally helpful to know your machine's hardware specs (processor, disk, memory, etc.)

What else might help?

The following are not always helpful, but can be. Use your best judgment!

  1. Session info; run sessionInfo() in R to get it. This tells us what platform you're on, your R version, and the packages that are loaded.
  2. An RStudio diagnostics report. Especially helpful if the R session is crashing or misbehaving. See Running a Diagnostics Report for instructions.
  3. An RStudio request log. Especially helpful if the UI is not working properly; this is a record of communication between the UI and R itself. Go to Help | Diagnostics | Request Log, press e to export, and copy/paste the results.
  4. Environment variables; run Sys.getenv() to see them.

Can you fix the bug yourself?

Here's where we remind you gently that RStudio is an open-source product, and we welcome contributions. If you have some programming experience, you can try your hand at fixing the issue yourself! We'd be very happy to help you do this (just ask when you file the bug).

The quickest way to get started building and running RStudio is with Docker:

Building with Docker

Thank you

Finally -- thank you for taking the time to read this, and to write a good bug report. It's an old open-source adage that given enough eyeballs, all bugs are shallow -- but that only holds true if those eyeballs are connected to hands that take the time to report what they're seeing! We're very grateful to the RStudio community for the hundreds of thoughtful, high-quality bug reports we receive every year.

Clone this wiki locally