• KubilayKocak
• Bugzilla
• Roadmap

Status: In Development (Request For Comments)

Scratchpad for laying out a roadmap for issue tracking & management within the FreeBSD Project.

Roadmap will not solely be a list of proposed features and changes. It is a master plan [1] documenting the primary purpose (its raison d'être), to create the basis for informing changes and improving decision making, independent of the issue tracking and management solution the project uses.

This roadmap is primarily motivated by and targeted towards bridging the wide information and expectations gap that exists for users (bug reporters, contributors, developers) of the system, specifically relating to their participation, value and impact in, and understanding of the process.

If you're interested in improving issue management within the Project, or would otherwise like to improve this Roadmap or other related documentation, don't hesitate to get in touch on #freebsd-bugs on Libera.Chat IRC.

Context

• Volunteer (Open Source) vs Commercial setting
• Externally facing (community interface) vs Internally oriented

Challenges

• Perception: Issue tracking "a chore", "not worth" reporting bugs
• Scale: More bugs, bigger queue
• Inaction: Lack of knowledge, experience, doubt or fear of doing the wrong thing (publicly) preventing action
• Consistency: Inconsistent handling, requirements, process by different people/groups creates dissonance
• Remit/Role: Overlap into "reviews / pull-request" type systems/workflows, other channels (eg: mailing lists for bug reports/discussions)

Purpose

The purpose of issue tracking/management, in any form, structured (issue tracking system), unstructured (email), formal or informal, is to facilitate effective product improvement

Other purposes include:

• Transparency: A public nexus for collaboration and communication
• Insight: Provides data about a projects product and practices
• Knowledge: Provides a history of who, what when, where, how and why. "If it's not tracked it doesn't exist"

Principles

Values

Goals

Potential Improvements/Changes

Automatic MAINTAINER approval of patches

1. We want maintainers to approve patches (attachments), not use comments or or the issue-level maintainer-feedback flag (as some currently do).

2. Bugzilla permissions are: anyone can edit a bug/field they have created themselves, or set a flag if they are the target of the flags ? <email> value.

3. Maintainers can't approve ports patches created by someone else, unless someone sets maintainer-approval attachment flag to ? first.

For ports issues with attachments, automatically set maintainer-approval ? <MAINTAINER-email> on attachments

• Issue: Not all patches in issues with cat/port in Summary are for that cat/port port.

  • Need: Parse patch, identify MAINTAINER-email
  • Idea: One hack is to potentially require and use 'cat/port' in attachment descriptions, and use that, otherwise...
    • Issue: Patch must be against ports tree root
      • Need: Can we codify for this? Eg: reject a patch if a known path/file cant be found?

    • This would also be useful in the general and src cases, to test if patches [still] apply, and do things accordingly (-- KubilayKocak 2021-06-02T02:04:35+0000 via MarkLinimon via WarnerLosh)

• Issue: Cant use this for base stuff, as it has a ... loose concept of maintainership. Most things/areas dont have machine parseable MAINTAINER lines
  • Need: Things in base to have MAINTAINER lines
    • Issue: Strict single-person MAINTAINER'ship is add odds with collaborative development, particularly in base
    • Issue: MAINTAINER'ship in base (as opposed to ports, which has a more obvious need for well-defined maintainers) may have negative effects ("territoriality")
      • Needs: More flexible definitions/schema for MAINTAINER lines (this would benefit ports as well) to include multiple people, teams, etc.

New Ports Component for new ports

1. Getting new ports into the tree is something we want to encourage
2. New ports take relatively more initiative and time to produce
3. Issues for new ports can often get lost in the noise of ports updates/bug reports

4. Resolution time for new ports is often substantially longer to commit than other change proposals, which can demotivate people

5. There is no consistent/comprehensive way to categorize new ports beyond Keywords: feature and Summary: [NEW PORT]
6. Summary tags like [NEW PORT] are user-set, and we are actively discouraging [tag] use in Summaries in favour of more structured metadata

So, create a new Component in the Ports & Packages Product, such as New Port specifically for new ports (ports that don't yet exist in the tree)

• Allows people who might be specifically interested in bringing in new ports to more easily find/work on them.
• Precludes the need for Summary [tags]
• Allows us to more easily see the 'queue' length without resorting to string parsing Summaries

Components with Default Assignees, also added to CC by default

1. Some Components have a specific default Assignee, either real users, or mailing-lists, to assist discovery of particular categories of issues

   1. Examples include: Ports & Packages Component issues with MAINTAINER's (not ports-bugs@f.o), Product::Base networking issues (Assignee: net@f.o)

2. In most cases, it is desirable for the default assignee to remain informed of issue updates even if they're no longer the Assignee

3. We presently do not automatically add Assignee's to the CC field, but we do it manually (particularly for examples above)

4. Accordingly, consider adding a Components Default Assignee to Default CC list too.

   1. Issue: We don't want to do this for "Ports" because ports-bugs is used effectively to mean Unassigned, and if it were on the CC list, every change for every issue would goto the list. SNR too low for that change

      1. Needs: Very clear definition of when to, or not to add Assignee to Default CC list.

More/Granular/Meaningful Components

1. The current list of components (in the Base Product) is highly repository/directory oriented, and is not granular enough.

   1. Eg: Product :: Base System kern and misc components are too widely-scoped

   2. Eg: Product :: Base System standards and arm components are ambiguous

2. Components are the primary categorization mechanism in Bugzilla

   1. Issues can live in only one component at a time.

   2. Components can have Default Assignee's and Default CC lists
   3. Components have descriptions (to help people figure out what to, and what not to) categorize there.

   4. Components are the primary way that issues can find themselves their most relevant (as a first step) audience (developers)

3. Components names should be as clear as possible to the user with regard to what issues should be reported there
   1. Reporters shouldn't require any knowledge about the project or its arcana (repo locations) to report into the correct component successfully
4. Components within a single a Product should be as semantically mutually exclusive as possible

5. Accordingly, come up with a scheme for more components in base

   1. Issue: the longer the list the harder it is for users to report correctly, and increases incorrect classification (and therefore SNR--)

      1. Chief arguments against infinite granularity: bad example #1; bad example #2

   2. Example: a component per directory/driver feels like too granular/much

   3. Example: a 'networking' (currently 'net') feels like not granular enough, though it is currently assigned to net@f.o mailing list

6. Some prelim scheme guidelines:

   1. things that aren't long-lived should probably be keywords
   2. things that can apply to multiple classes of things should probably not be components
   3. Widely-scoped components could potentially be separate products with their own components, where it makes sense.

      1. Eg: Product: Device Drivers Component: <list of drivers>

      2. Eg: Product: Firewalls Component: <ipfw, pf, ipfilter>

7. Confusing existing examples:
   1. conf, standards (can apply to any component), gnu
8. Likely Examples:

   1. "Linux Emulation" a major 'feature' of Base. Has an actual, known or effectively maintainer or set of committer(s) that are familiar with that area.

   2. Toolchain, ZFS, FuseFS (or Product::Filesystems Components: ZFS | FuseFS | UFS | etc
   3. Installer

   4. Audio & Sound

   5. Virtual Memory Subsystem
   6. Network File System (NFS)
9. Maybe/Unsure Examples:

   1. Netmap, DTrace, Bluetooth, GEOM, IPFW/PF ("Firewalls"?) IPFilter (has a maintainer), Jails, PkgBase (not an ongoing 'base' component?), RC, ACPI

   2. Project specific, potentially with dedicated people/teams: Reproducibility
10. What to do with:

    1. "tests". Testing & CI is a project team with its own infra, processes, not strictly 'tests'. Possible rename "Testing & CI"

       1. What about a future unified Testing & CI team across repo's (in particular base/ports but also docs)

    2. 'freebsd-update' (the command). Major user feature, single binary. Different from Services::FreeBSD Update (the service, and infrastructure)
    3. Specific projects, like MAC, TrustedBSD, libxo etc that can touch many other component code

Other Notes / Thoughtstream

1. Maintainer change / reset -> reset/reassign existing assigned issues

2. Consider Send Upstream style resolution: less blunt than Not Accepted, not really satisfied that existing resolution are more appropriate (?)

3. The Severity field values (Affects: Only Me, Some People, Many People) are not great. Come up with a better scheme that is meaningful and not arbitrary (ala 'Low/Medium/High')

4. Generic feedback  flag, issue and attachment level. Maybe attachment one is called review. Currently we only have a maintainer-feedback flag, which we sometimes use to ask for feedback from reporters or committers (who are, ostensibly, not-maintainers), which can confuse some people.

   1. attachment review flag needs to be considered in tandem with needs-qa which implies similar, but without a specific target/person to whom it is directed (unlike flags)
   2. Improve Auto-Assigner UX in certain cases
      1. MAINTAINER doesn't have Bugzilla account with that email. Ask maintainer to make them match, make AA comment better
      2. Unmaintained ports, add comment with messaging about considering becoming a maintainer

   3. TriageBot

      1. Auto search stale (no changes in <period>), add issue is stale comment, close requesting re-open if still an issue

      2. Auto search assignee timeout: assigned to @freebsd.org and (no changes in <period>, reset Assignee, add comment

   4. On Status->Close - Assign issue to closer, unless already assigned to !Nobody

5. Automate 'Maintainer timeout' "pings" (reminders) on issues

6. Often times, the Auto-Assigner cant determine a maintainer. In this case, Triage needs to identify the issue, lookup maintainers manually (ports or otherwise), and update Assignee/maintainer-feedback. It would be nice to reduce the manual'ness of this where we can and it doesnt create more issues. 'Re-Run AA' might be something that can help (-- KubilayKocak 2021-06-02T02:11:42+0000 via MarkLinimon)

References/Addenda

See Mozilla's main Bugzilla Products (note WebExtensions is a Product, not a Component of Firefox)

Note: How Mozilla Product:Core Components are faking another layer of separation with "Foo :: Bar" in component names