rpiguy
Posts: 25
Joined: Mon Mar 26, 2012 10:01 pm

Centralized bug reporting?

Wed May 16, 2012 1:44 am

As a community, we should have a centralized place to report bugs. The forum isn't a great choice for this.

I have no idea what software we should use. Trac and Redmine spring to mind, just because they're used by a lot of projects with self-hosted bug reporting.

I also have no idea whether this should be hosted by the Foundation, set up by some kind outside volunteer, or if we should use some third-party solution (SourceForge? GNU Savannah? Google Code? Github?)

Finally, I have no idea if this has been / is being / will be publicly discussed, or if I'm the first to ask the question.

Here's an example of a bug reporting system:

https://bugs.launchpad.net/ubuntu/+sour ... bug/944546

Please post your thoughts here.

rpiguy
Posts: 25
Joined: Mon Mar 26, 2012 10:01 pm

Re: Centralized bug reporting?

Wed May 16, 2012 1:44 am

The most important things a bug report system has are:

(1) Voting on bugs. Bugs that affect more people can be efficiently prioritized.

(2) Shows the current status at the *top* of the page; up-to-date information in forum threads often requires reading the entire thread which may have many pages -- if you can even manage to find the right thread of possibly several on the same topic

(3) Bugs can be marked as duplicates of other bugs, or related to other bugs. This makes it easy to redirect discussion to a single place, and makes it easy for users to find where the up-to-date information is (forum threads have no way to deal effectively with fragmented discussion).

(4) In bug reports, mods should delete or move posts which are not directly related to a single bug, keep discussion on each bug's page focused on that particular bug. Many forum threads are filled with off-topic posts which just get in the way when you're looking for one specific thing. Mods can and should have a lighter hand with off-topic posts in the forum, but bug report comments need to be held to a higher standard of on-topic-ness.

(5) Bug reports tend to create very concrete, clearly delimited, testable objectives for developers.

(6) Bug reports create a single, clearly marked place for users to report problems.

(7) Bug reports create a single, clearly marked place for people to share workarounds with each other and developers.

(8) Bug reports can be easily linked from forum posts or other websites.
Last edited by rpiguy on Wed May 16, 2012 1:51 am, edited 1 time in total.

rpiguy
Posts: 25
Joined: Mon Mar 26, 2012 10:01 pm

Re: Centralized bug reporting?

Wed May 16, 2012 1:44 am

This thread was inspired by the following thread ("First impressions - Slow, Scary & unfriendly" by bamboozled in Absolute Beginner forum):

http://www.raspberrypi.org/phpBB3/viewt ... f=5&t=5506

The above thread is a multi-page mess, which opens with a post detailing many different problems the author had starting his Pi. Most replies only address (at most) a single bug. Of course, there are plenty of ad hominem flames (mostly of the form "the information is somewhere on the forum/wiki/blog comments/Internet, you should try harder to find it," or "if you really hate your Pi you should sell it"), and others defending the author. The thread is unfocused and difficult to follow.

I don't mean to pick on the OP of that thread or its other participants. I'm sure most of them posted in good faith, and there were a fair number of posts that I'd actually classify as productive discussion. The problem is that it's difficult for someone coming to that thread, who's maybe only interested in one of those problems, to get only the information they want without being forced to follow what is, to them, several other discussions of completely irrelevant bugs, not to mention a flamewar. Add to that, the status of things may change over the months and years as the software evolves, other complaint threads will surface talking about the same or similar problems, and it'll be very difficult to determine -- even a few months from now -- which, if any, of the discussions reflects the current situation.

Even though it may seem that I'm attacking the above-referenced thread, I'm actually saying that it contains some valuable discussion -- my complaint is that our current online tools aren't best way to organize that valuable discussion in a way that best facilitates constructive progress.

So I don't have a problem with the members of the forum community. I don't have a problem with the idea of having forums, or with the new forum software. I'm just pointing out the simple, well-established fact that forums aren't an effective medium for bug reporting.

The best evidence I can give that this fact is well-established is that most major open-source software projects -- and even many minor software projects -- have and use both a forum and a bug tracker. That's because both forums and bug trackers are necessary for their communities to maximize productivity.

Our community should have both, as well.
Last edited by rpiguy on Wed May 16, 2012 2:02 am, edited 1 time in total.

shirro
Posts: 248
Joined: Tue Jan 24, 2012 4:54 am

Re: Centralized bug reporting?

Wed May 16, 2012 1:58 am

Currently firmware and kernel bugs are being reported on github which seems to work for the current volume and a developer focus.

I think perhaps something like launchpad might make sense as the community grows. Though possibly Raspbian and Arch and others will use their distributions chosen bug reporting system and the distros package maintainers will file the bugs upstream so a centralised system might not be needed.

rpiguy
Posts: 25
Joined: Mon Mar 26, 2012 10:01 pm

Re: Centralized bug reporting?

Wed May 16, 2012 2:41 am

shirro: Certainly many bugs will be duplicates of upstream ones. And there will probably be plenty of linking back-and-forth between our bug tracker and Debian's.

But there are surely going to be plenty of issues that are Pi-specific. For two examples from the OP of thread [1]:

(1) The underscan issue is specific to the Pi hardware and firmware, and its solution involves the OS-agnostic bootloader file, config.txt
(2) Whether to have automatic login enabled by default (i.e. whether to require a username and password to login by default) might have a different answer for ourselves and Debian. Debian users and RPi users (especially after the educational release) are communities with totally different levels of computer experience. Not to mention Debian has a years-long tradition of requiring a login that will upset a large installed base if changed, while Pi can make a different choice because it's new enough it has no established tradition yet.

Anyway, the point is that we're not just using regular Debian; we have custom hardware, custom (proprietary!) bootloader, custom drivers, our own selection of default packages, and our own scripting (or likely will have these before the educational release).

We also have our own community and culture -- I feel that we have more hardware folks than Debian, fewer software/applications/IT folks than Debian, and (assuming the Foundation has success getting the Pi into schools) we'll have a lot of users who are educators and K-12 students, whose needs are considered to have high priority by much of our development community.

In short, the RPi project as a whole is distinct from both Debian and the Pi kernel/firmware development on Github. We use software from both projects extensively, of course, and many bugs that exist in the distro are surely inherited from one of them and should be reported and fixed upstream at the source.

Consider Ubuntu. It's a Debian derivative with lots of customization to support a distinct community with different objectives than Debian. Ubuntu bugs are often actually Debian bugs, or bugs in particular packages, but that doesn't mean Ubuntu shouldn't (or doesn't) have and use its own bug tracker. Ubuntu also certainly produces its own bugs through its customizations.

There's some value in a centralized reporting place, too. Is this a kernel bug? Is this a bug in a particular package? Is this a distro-related scripting / packaging bug? Is this bug Ubuntu-specific, or does it only exist in Debian? An end-user should be able to report it somewhere without being able to answer these questions. And then someone more knowledgeable can take on the task of figuring out which upstream source it belongs to, if any, find or file an upstream bug report, and link to the upstream bug report in the downstream report.

Replace "Ubuntu" with "the canonical RPi Debian distro" in the above description, and you have a decent argument for why we should have our own bug tracker :)

[1] First impressions - Slow, Scary & unfriendly - http://www.raspberrypi.org/phpBB3/viewt ... f=5&t=5506

rpiguy
Posts: 25
Joined: Mon Mar 26, 2012 10:01 pm

Re: Centralized bug reporting?

Wed May 16, 2012 3:00 am

Also, part of the reason our community seems so small, from the standpoint of bug reporting, is that very few of those who have ordered have gotten an RPi yet. It's very hard to report or fix bugs if you don't have access to the hardware.

I have a feeling there's going to be a deluge of bug reports when the number of shipped Pi's starts to pick up, which will turn into a flood with the first educational users.

I think things would be better for us if we had an online bug reporting tool in place when that happens. Notwithstanding several flamewars on the subject in these forums, I would say that many people actually do understand "This is a new thing, there will be problems," and will be somewhat tolerant of bugs in the beginning.

But if we don't have an effective way to manage our handling of their problems, if we just "pass the buck" to Debian, which has different priorities and a different culture than we do, we risk alienating users. I daresay the Pi will always have a place with hackers, but they're not the intended audience -- or at least not the only intended audience :)

Likewise, the kernel issue tracker on Github is definitely a good thing -- but I foresee having many bugs that aren't kernel bugs. The kernel, while important, is definitely only a part of the Linux distro -- a distro is actually a complex collection of applications, libraries, configuration, and scripting, and any of those components (or their possible interactions) can produce bugs. Kernel issues are important and the mechanism we currently have for handling them is a good thing -- but I foresee many end-users having problems that have nothing to do with the kernel.

rpiguy
Posts: 25
Joined: Mon Mar 26, 2012 10:01 pm

Re: Centralized bug reporting?

Wed May 16, 2012 3:25 am

I mean my comments about "RPi bug tracker" to apply to the One True Distribution that has been "blessed" by the RPi Foundation.

That distro should have the One True Bug Department where users can report any sort of problem they have, and a Guru will see their complaint and Do Something about it.

That's the end-of-story for the end-user.

In reality, of course, the Guru has to do a little legwork to figure out which component of the system is causing the faulty behavior, figure out the proper upstream source (if any) for that component, then cooperate with their developers and bug system.

Making our end-users do that kind of legwork is simply unrealistic for the intended audience.

Trying to make them do that kind of legwork will end badly for them (they'll have a very negative experience trying to get a response to their real problems) and for us (the Pi will get a reputation of "user-unfriendliness" and will become a device limited to a niche audience of fanboys and tech enthusiasts, instead of introducing Linux and programming to an entire generation).

If non-"blessed" distros -- major players like Debian, Arch, and Gentoo, or derivatives thereof, or derivatives of derivatives -- want to officially support the Pi hardware platform, including kernel, and provide their own bug report mechanism, I have nothing against that.

It's just that the massive influx of nontechnical users who want a "friendly" distribution and use the "blessed" one, should have a unified place to send bug reports. And AFAIK the "blessed" distribution isn't a Debian that's supported by Debian; it's a Debian derivative, and hence, can have its own bugs that don't exist in Debian and may not be tolerated on the Debian bug trackers.

hippy
Posts: 12485
Joined: Fri Sep 09, 2011 10:34 pm
Location: UK

Re: Centralized bug reporting?

Thu May 17, 2012 12:26 pm

rpiguy wrote:But if we don't have an effective way to manage our handling of their problems, if we just "pass the buck" to Debian, which has different priorities and a different culture than we do, we risk alienating users.
Agreed; R-Pi users shouldn't care why there's a bug or who will deal with it and there should be no buck passing. They need a one-stop shop for all R-Pi issues.

If a bug is inherited from upstream it must continue to exist as an open R-Pi bug until whoever upstream fixes it and it's rolled out to the R-Pi community. Only then marked as resolved.

User avatar
williamhbell
Posts: 291
Joined: Mon Dec 26, 2011 5:13 pm

Re: Centralized bug reporting?

Tue Jun 19, 2012 10:28 pm

Hi,
Asb suggests we use https://github.com/asb/spindle/issues for now.
Regards, Will

Return to “General discussion”