compiler-team Documentation

https://github.com/rust-lang/rust/issues/64343

2019 roadmap (for reference)

The compiler team has five main themes of work for this year:

Improving "core strength" by lowering raw compilation times and also generating better code (which in turn can help with compilation times)
Improved IDE integration, including support for typed completions
Extracting parts of rustc into libraries, which are easier to understand and maintain but also help in developing a Rust specification
Supporting lang team in key feature development, such as around async fn support, or improvements to the trait system
Improving our documentation and organization, helping to make it clearer what is going on at any given time, and how people can get involved.

Big challenges for rustc and compiler team

Poll to vote -- feel free to add your own items to the poll; you can also add notes below with a few lines of elaboration.

compilation times remain stubbornly high
- many companies have huge codebases and compilation time can be an existential crisis
- we feel this ourselves with bootstrapping
  - band-aids like "factor librustc into finer grain crates" might work; cannot be "official answer"
- incremental has helped but it's not like a 1-line change is 'instantaneous'
- this bullet is talking about the time it takes to "run tests" and do full builds; it does NOT refer to the "instantaneous feedback" for things like completion, which falls under IDE
tech debt around the trait system
- blocking a lot of lang team progress (async fn in traits, GATs, specialization)
- "lazy normalization"
  - also maybe blocking const generics work
  - also support for higher-ranked lifetimes etc remains a common place where you can hit ICEs that have no real fix
- possibly performance impact
maintenance and triage work
- we do a decent job of keeping up with work but work burden is unevenly distributed
- non-urgent / non-regression bugs can "pile up" -- are we making progress on them?
- (note absence of data here; maybe we are keeping up, relative to bug-report rate and/or bugs that actually matter...)
  - (simulacrum): we are not really keeping up
  - long-term trend is up; most current issues are fairly recently triaged,
    count has been going up: ~3000 last summer, up to 4900 now. Not all compiler team, but "most".
IDE story is confusing, we're not focusing our efforts
- we need a strong IDE story
new features draw in new volunteers more easily than maintenance/paying-down-debt
- can we make such work more exciting? E.g. via gamification?
documentation and design
- many parts of the compiler remain undocumented
  - writing mentoring instructions often requires recounting same details over and over
- we have a lot of great design ideas but there are often ill-documented
- ambitious rewrites can be hard to review without knowing 'the plan'
  - also, often it's not clear how much consensus there is on the plan
paying for labor
- we rely on volunteers for vast majority of coding effort
- some of this is coding done while "on the clock" for some boss, with that boss's blessing
- (there are also contract laborers, financed by companies with specific needs)
- is this model of compensation going to suffice going forward?
team growth
- Find more people to help out
- Improve the onboarding process
- Mentoring/help

Discussion summaries

compilation times

benchmark vs end-user programs
- maybe set a strict goal of improving bootstrap times (link)
- rough conclusions
  - bootstrap is worth focusing on
  - codegen time is a big deal
    - measurements suggest librustc is long pole in the tent, also
- incremental could be improved, e.g. measure/improve the "edit comment" use case
  - tracked by rust-lang/rust#47389
  - eddyb writes:
    - don't do any work past the HIR when only a comment changed (targeted Span improvements may help)
  - move incremental back so you don't need to spend entire seconds getting to HIR

trait system

made progress this year but not as much as hoped
conclusion:
- we need most of all a clear roadmap here, and that should be an immediate focus
- good to try and separate out lazy normalization from other goals

File: content/minutes/design-meeting/2019-10-11-DepGraph-persistence-PR62038.md

title: Dep-graph loading (#62083) type: docs

design meeting 2019-10-11

Zulip stream

introduction

Dear @T-compiler/meeting,

Today we will be having a design meeting. The topic was originally sketched as "some Zoxc PR". We've since narrowed that down to discuss #62038, which is a refactoring to how dep-graph loading occurs. @Zoxc wrote up a comment giving a summary of the ideas. Note that this PR itself is an incremental step towards #60035, which aims to make dep-graph loading/saving more continuous.

I'd also like to discuss briefly how we should document these changes. We currently have some rustc-chapters on incremental compilation (e.g., this chapter goes into detail). I would like to move us to a world where major refactorings like #60035 (but not limited to this one -- I think e.g. my recent PR and work on lazy norm fits the bill) come along with a rustc-dev-guide chapter that documents the new state of the world. Maybe we discuss some how that might work and -- in the case of THIS PR -- who might do that documentation work (I don't necessarily think it has to be @Zoxc, though they're also an obvious candidate). (In my ideal world, drafts of that chapter would be available before the PR, but at minimum I think such a chapter should be in place to help with reviewing.)

Questions for discussion

"This is where the performance gain of this PR is.", have we measured this at all? -- nikomatsakis
- Here are some measurements. However, are they done against a single-threaded or parallel compiler? -mw
- The results show some improvements and some regressions.
"The one possible performance drawback is that ids can become fragmented since this PR requires us to reuse ids from the previous session." -- in what way is this a performance drawback? Do we have any mechanism to reuse ids or 'reset' state after a suitable amount of time?
There is another possible performance drawback: The current dep-graph has to be built two times in the worst case (i.e. full cache, but all invalid). Is that correct?

Minutes and notes from discussion

Review of how existing dep-nodes work
Questions about the PR
- Key idea: change is to load up the old graph and use it as the starting point, editing it in place, rather than copying things out from it into the new graph
- Deleting nodes and garbage
  - when we load nodes, they have no color
  - if at the end of compilation they still have no color, we can delete them
  - we keep a "free list" for indices so they can get re-used
- However:
  - we never shrink the graph as a whole, so if there used to be a lot of ids, we will keep the graph the same size
    - in particular there could be an exceptional execution that creates a lot of nodes
  - ids may also become fragmented over time
- How could we address this?
  - some form of "compression" step when writing back to disk
  - complication: query-result-cache uses ids as keys
    - "if we want to garbage collect them, it must be done in sync with the query result cache"
Conclusions
- We had a good discussion of the PR in question. I don't think we raised any red flags or anything, the approach seems solid.
- The real question is whether we want to move in he overall direction proposed by #60035
- The goal here is to reduce the cost of loading/saving the dep-graph
- #60035 proposed to do so by incrementally dumping out changed nodes and not retaining them in memory
- This we barely touched on -- some open questions for me
  - Are there alternative designs we have in mind?
  - Is #60035 itself an "end state" or a stepping stone?
- Conclusion:
  - Follow-up meeting to dig into design of #60035 and maybe discuss alternatives
  - In parallel, Niko will review #62038 now that he understands roughly what it is trying to do
    - We would also want to write docs for rustc-dev-guide
    - We can figure out how that happens in parallel

File: content/minutes/design-meeting/2019-10-18-debuginfo-meeting.md

title: Debugging Support type: docs

Debugging Support in the Compiler

Zulip stream or read on the Zulip archive

The problems

We have bugs with debuginfo that are causing broad pain for people who use x.py test, but we do not have dedicated developers who own maintenance of debugger support.

https://github.com/rust-lang/rust/pull/60826

We have Pull Requests to improve our debuginfo support, but we do not have dedicated developers who own maintenance of debugger support.

Q: Are we willing/able to maintain debuginfo stuff Q: if we don't, then can we afford to keep these tests?

In short, as Niko said: "we need help to maintain this or we may have to remove it"

The overarching goal was to discuss how we can get a sustainable story than the particular issues themselves, though that may also be a topic of discussion.

what steps can we do to recruit folks
what are some thresholds and timelines where upon we consider more drastic action
what might that drastic action be :)

The meeting

some of the main points from the discussion:

a common refrain: most of the compiler developers know next to nothing about how the debugger support scripts are implemented. So we are not in a good position to maintain the debugger support scripts.
@eddyb has ideas for how we might try to shift functionality out of the debugger support scripts and instead "handled it through traits and codegen" (?)
debuginfo tests are effectively unmaintained because everyone has their own ways of ignoring the failures
- We should add // ignore to the problematic tests in the short term (but make sure to associate any such ignore with a rust-lang/rust issue).
we might want to split the tests into debuginfo and debugger-pretty or something; the debuginfo stuff we must keep working, while debugger-pretty are more "nice to have" bits of functionality

File: content/minutes/design-meeting/2019-11-01-streaming-dep-graph-proposal.md

"Streaming dep graph" proposal

Summary of the proposal

Currently, incremental compilation begins by loading the old dependency graph and then -- as execution proceeds -- building a new dependency graph in memory. At the end of compilation, this new graph is written out.

PR #60035 proposes to change how the new graph is handled. Instead of building the graph in memory and writing it out at the end, we "stream" out a series of changes to the dep-graph as we go. When parallel compilation is enabled, this streaming can proceed in a background thread.

Advantages

This has the following advantages:

Lower peak memory usage
- e.g. style-servo-check has a 30% reduction in memory usage in the May 7th perf results and there are no cases of memory usage increasing
Avoids blocking on writing out the dep-graph at the end of compilation, which improves performance, especially for smaller compilations
- the magnitude here is a bit unclear
- the May 7th perf run shows up to 8% regressions in instruction count, but up to 5% wins in CPU time (and wallclock time), perhaps related to parallelism? --Niko. That is because loading the dep graph is slower, but it happens on a background thread.
Avoids the work of re-writing unchanged portions of the dep-graph.
* (pnkfelix notes that in the most extreme cases, e.g. of rebuilding a touched but unchanged files, this can be significant.)

Dicussion of goals from Zulip conversation:

starts around here
writing out the dep graph occurs in parallel with LLVM right now
- but if there is nothing for LLVM to do, that doesn't help
clean incremental runs ought to be "worst case" measurements for current system
- no changes, nothing for LLVM to do to mask latency
existing measurements suggest:
- up to 30% wins for peak memory usage -- example, style-servo-check
- up to 9% wins for CPU -- example, regex
motivating example:
- unused-warnings-check, where clean incremental costs more than clean

PR Thread

Some key comments from the PR thread (which is long)

perf comparisons (I'm not sure on the differences between these -Niko) (Only the last one has a GC, but I don't think that perf triggers that anyway. Also the PR is still WIP so perf is probably not completely representative -Zoxc)
mw expresses concerns
- generally, that the change looks potentially quite good, but adds a lot of complexity, and we need to have good docs and more discusion -- this meeting is part of that!
nnethercote advocates in favor on the basis of memory usage
- "A Firefox developer was complaining recently that compilation of Stylo within Firefox was causing OOMs because memory usage was exceeding 3.5 GiB. I did some profiling runs with Massif and DHAT, and for incremental builds the dep graph dominates the memory profiles -- most of the top entries involve the dep graph."

Questions regarding implementation

What does the on-disk and in-memory format of the data look like?

A great answer to this question would be something that can be used as module-level documentation for

https://github.com/rust-lang/rust/blob/master/src/librustc/dep_graph/serialized.rs

In particular, if we are only saving updates, do we have to do more work each time? We do have to do more work when loading the dep graph as garbage is accumulated. How do we collect old data?

Zulip discussion from meeting starts here
On-disk format is a list of actions
- each Action is read in and "applied" to an empty dep-graph when rustc starts
- note that some actions may overwrite the effects of previous actions
  - comment
How do we manage the allocation of edge vectors?
- current PR uses Box<[Index]> to store edges, but this can be wasteful
- old approach of using a single vector of indices was efficient but hard to update
  - but we could just waste the space and rely on a GC
- could move to arena allocation (e.g., &'tcx [Index]), though it shares some downsides with edge vector
Maybe we can read the data in backwards to avoid the update?
- maybe, would have to work on the serialization format
- currently using LEB128 but this is not obviously the best
  - and there were some speedups from avoiding it elsewhere

What form of garbage collection do we do

As we are storing updates, there can be garbage data of two kinds:

Graph nodes that were overwritten with a new set of dependencies in the latest compilation
Graph nodes that were never used in the latest compilation (e.g., type-checking nodes for functions that were deleted)

How do we remove these two kinds of old nodes? Is this mechanism conservative or precise?

The garbage collection algorithm reads the dep graph and records the location of information we want to keep. It then clears the file and writes only the useful information back. It is precise and happens in the background thread. The cost of garbage collection is similar to loading the dep graph, so it's unlikely we'll block on it.

discussion in Zulip around here
- current GC just removes actions from the list of actions
- it does not compress DepNodeIndex ranges
  - this implies that the size of the graph will grow if fragmentation occurs
  - but we do make an effort to re-use dep-node-indices
GC comes after loading and become new actions can be appended
full pipeline including GC (link)
but how do we remove "unreachable" nodes?
- they wind up with no red/green color at the end of compilation
- and we can iterate over the nodes and "remove" those that have no color

Tradeoffs

What will happen if/when we want to move to a rust-analyzer like model, with live updates?

It seems like this model is just incompatible with that. I think this implies that we would want multiple 'modes' of operation.

Does this model imply an in-memory dep graph only? -Zoxc

Are there performance regressions to be expected with this approach?

There's a possibility of dep node indices getting fragmentated. This is shared with #62038.

Are file sizes in the same ballpark as before?

They start in the same ballpark and will grow from there. We can control how large they grow by how often we run the GC. We can also throw in some compression to get then down further. We won't have to pay for the cost of compressing the entire dep graph each time, only changes. We still have to decompress the entire thing, but that happens in the background thread.

Have other approaches been evaluated? What are they?

One obvious alternative would be to stream the entire graph as it is updated, and not deltas. This sidesteps the GC issue, but has the same shortcomings as our current approach when it comes to 'partial updates'. I think this PR started out as this, but then it grew a bit -Zoxc

mw raised an idea in the design meeting (link):
- (I'd be interested in how an alternative approach (that also does DepNodeIndex compaction and graph node GC) does: Still rewrite the whole dep-graph, but "stream" it to disk in the background. That should be doable with minimal changes and might have most of the benefits.)

Is there an approach that enables less processing being needed?

Perhaps storing a list of where dep node indices are stored in the file. That would allow the loading to skip to the relevant parts of the file and avoid the garbage slowing down as it accumulates.

Future features

Does this approach scale to partial updates of the dep-graph?

Definition: "Partial updates" refers to smaller incremental runs that don't re-perform the entire compilation. This can occur in the case of compilation errors. It also occurs in an IDE scenario, where e.g. the latest query may be to find the completions at a particular point. The challenge here is that there may be parts of the previous compilation that were not required for the particular query that was just executed, but which would still be worth caching in the future. Our current setup doesn't handle this case.

I don't see a reason why not -Zoxc

Open Questions

how much of a problem is dep-node index fragmentation?
does this resolve the perf issues around unused-warnings performance that pnkfelix described?
is there a "reasonably small diff" that modifies current design to stream out graph and gets a lot of the benefits, as mw suggested here?
- the basic idea would be: instead of building the new dep-graph in memory and encode it in the end, have a background thread that we send new nodes to and the writes them to disk. Everything else would be the same: The entire file would be re-written, including the unchanged parts. The on-disk format would maybe change in that there would be a seperate file for each of the 4 arrays in SerializedDepGraph, so they can grow independently. Alternatively (but this might be a bigger change), there would only be a single file with variable sized (DepNode, Fingerprint, [DepNodeIndex]) entries.
- this approach should also lower memory usage and there is no DepNodeIndex fragmentation, but it also does more work than an append only approach

File: content/minutes/design-meeting/2019-11-09-unified-dataflow-framework.md

title: Unify Dataflow Frameworks (#202) type: docs

Design Meeting 2019-11-08

Zulip Stream

Agenda

Design doc: https://hackmd.io/@39Qr_z9cQhasi25sGjmFnA/Skvd9rztS
Explain the current state of things
- We currently have two wholly separate APIs for dataflow, one for bit-vector problems and one that allows arbitrary transfer functions but still restricts the lattice to a powerset of indices.
- I want to merge these two to make code more DRY.
- The prototype implementation is described pretty thoroughly in the design doc, but maybe go over it a bit?
Discuss the prototype implementation in general
- The prototype is not very ambitious. It mostly preserves the existing BitDenotation API but extends it to arbitrary transfer functions.
- Do we want to maintain the status quo? Have two separate implementations forever? Once arbitrary lattices are supported, maybe they won’t fit as nicely into the same API.
  - OTOH, adding backwards analysis would have to be done in two places.
- There have been more ambitious changes attempted. Perhaps we want to do one of these instead? Or at least incorporate some of their ideas into a framework that hews closer to the existing one.
  - nagisa worked on a new dataflow framework that works for arbitrary lattices and transfer functions back in 2016.
  - eddyb was looking into a simultaneous forwards/backwards dataflow-like thing for NRVO.
Discuss the prototype in detail (If we decide we still want to go this way)
- How do we feel about the {before_,}*_effect naming convention? It takes a bit of getting used to, and doesn’t extend nicely to backwards dataflow.
  - OTOH, having a canonical statement_effect which most implementers want to use and a secondary, prefixed effect which is only rarely used is good.
- Should we be passing a mir::Statement to the statement effect methods? What about mir::Body to bits_per_block?
- Should we use specialization to implement this?’
- Should the Analysis methods take &self or &mut self?
  - Some may depend on the results of an earlier dataflow analysis and want to hold a cursor.
  - However, transfer functions should be pure for a given MIR, so maybe forcing those analyses to use interior mutability is appropriate.
- Are we okay with a ResultsCursor becoming the common way to inspect results?
- How do we implement a zipped ResultsVisitor?
Discuss what extensions to the framework we want to prioritize
- Backwards dataflow (currently implemented by hand to do check liveness in generators)
- Arbitrary lattices (would make MIR const-propagation more powerful)
  - OTOH, the current linear const-propagation gets most of the benefits without the overhead of dataflow.
- Optimizations:
  - Don’t cache block transfer functions on acyclic MIR (reduces memory usage)
  - Extended basic blocks? @Mark-Simulacrum

Minutes

Background on the need for arbitrary transfer functions for dataflow and const qualification.
Alternatives to the prototype:
- Perhaps we could express these more advanced dataflow problems in datalog/datafrog?
  - nikomatsakis wanted to test more in the polonius repo before committing to this upstream.
The need for apply_call_return_effect:
- We need some version of this, but not necessarily the current version.
- API details
  - Skipped most of these, since they can be figured out later.
  - mir_borrowck might benefit from ResultsCursor now that it is implemented.
Optimizations/extensions
- Some interest in not caching block transfer functions for acyclic MIR.
  - This is pretty easy, I will prioritize this.
- Probably not going to do EBBs anytime soon.

Conclusions

There was consensus on pursuing some version of the prototype instead of something more radical.
- Details will be hashed out in a PR to come.
- @ecstaticmorse will pursue this.

File: content/minutes/design-meeting/2019-11-16-Working-Group-Retrospective.md

title: Working Group Retrospective type: docs

Working Group Retrospective

Zulip stream or read on the Zulip archive

What you are looking at

We did a short survey before
the meeting. The following notes were scraped from the 10 responses. I
tried to de-duplicate common items. Towards the end, you will find
some minutes that were taken during the meeting itself. --nikomatsakis

Things to KEEP doing

Working groups are useful
Nice to know who to talk to in order to get involved
- this was more of a struggle before
Providing opportunities for mentorship is good
Scoping out small groups of decision makers for "in the weeds" decisions
Weekly updates
Dedicated Zulip streams: great to have ability to link/skim, even if most are muted
Regular meetings are a good way to keep working groups focused

Things to STOP doing

Maybe fewer working groups, with more time/energy invested in each?
- Some of the smaller WGs don't see much activity, could discourage people

Things to START

Check in with WGs not just about achievements but process
- e.g., diagnostics and mir-opt don't have any meetings
- Question from editor: Not clear if the idea here is to try and have a more uniform process across teao use that as a signal that help may be needed
  - yes, that's what I meant
    - which is what you meant? :)
Improve our labeling system
- Tag PRs with WG-XXX instead?
  
  [name=Centril] Instead or both? We (T-release) can still use A-* labels.
  [name=Felix S Klock II] I worry about switching to this (rather than the A-categories) because it can mean that multiple WG's get pings for incidental stuff...? I guess the labels don't actually cause pings so maybe that doesn't matter. Another potential issue (again that arises from multiple WG-labels on one issue) is that it can become ambiguous which WG should be taking lead on something.
Proactive recruitment for specific working groups?
Devise metrics to evaluate effectiveness?
- [name=Centril] What would those metrics be (also... performance reviews? ^^ Rust is not a company; heh)
- Should WGs create those?
Figure out the "wind down" process (e.g., for groups like NLL and pipelined compilation)
- Do we need to document which members are responsible for long-term maintenance?
- Consider e.g. NLL
- How to ensure the remaining work gets done? A lot of it is falling to Matthew Jasper for NLL, is that ok?
Some better way to advertise what we are doing

Things to IMPROVE

Longer timeslot in compiler team triage meeting (maybe just a few minutes)
- More time for discussion or Q&A related to plans, progress, etc
  
  [name=Felix S Klock II] maybe if we just moved it to the begining but had a hard limit on time?
Onboarding could still be improved
Some better way to get a sense of status without participation in every compiler team meeting
- Maybe use Inside Rust blog or other means to make more regular public announcements
- Maybe a regular (but less frequent) time when all groups can check-in
Require explicitly documented leaders, some older WGs are missing them
Improve use of WG labels:
- the set of github WG labels and actual WGs do not correspond in a meaningful fashion
- Hard to know when to use the A label vs WG
  - [name=Centril] This is primarily a matter for T-release triage. When in doubt, use both labels.
Distinction between shorter and longer term groups
- maybe use distinct terms
- add metadata indicating this
Help ensure that each working group has active leadership, which makes a big difference
- hard to find people with enough time
- maybe we can help leaders?
- multiple leads are good
  - [name=mw] +1! [name=Centril] +1
More involvement from the lang team on Zulip is useful, developing a better working relationship between lang and compiler team would be good
- [name=Centril] The lang team recently "moved" to Zulip
- [name=Centril] Narrowing the funnel through which T-lang adds new work to T-compiler's queue would be good
Relationship between triaging wg (from release team) and compiler team
More retrospectives to help us reflect on how things are going!
compiler-team web pages feel like they could be better
- templates are not that useful
- would be nice to have the updates there
overhead of maintaining all the web pages and other things feels too high
- can we post updates in a more central fashion?
fold the activity of "regular contributors" into the working groups more

Other things

don't generalize too much from NLL, it was a big prominent lang feature with lots of interest

notes and ideas from meeting

The following notes and conclusions were extracted during the meeting. For more details,
please see the complete Zulip stream.

discussing check-in and how to keep the info up to date

maybe a zulip stream that people can post updates to
- perhaps with bot support to scrape and collate
- this would land in github in the form of meeting minutes
- but it would also make it easy to catch up on the status
make a point to help WGs develop roadmaps
- helps to frame check-ins
metrics?
- but what? and let's not try to make this too much like a company
wind-down
- things like NLL are winding down but it doesn't feel very organized
- even the concept of wind-down requires more work on roadmaps, so this ties in with the point above
- maybe the next step for winding down is to try and use that process, but aimed more at finding the work left to be done
  - goal: "what did we get done, what's left, and -- if it matters -- who will do it"
kinds of working groups
- groups doing manual, recurring tasks and upkeep
- groups around an area of the code ("WG-NLL", "WG-mir-opt")
- project groups with a pretty clear goal ("WG-NLL")
  - maybe projects are things that "area-based groups" do?
- groups to ping people ("ICE-breakers", "WG-rollups")
labels
- some way to indicate that the issue has been triaged by a given wg (e.g., AsyncAwait-Triaged)
follow-up topics to drill into more
- creation of wgs is undocumented
  - there are ongoing activities that it would be great to "capture" in a WG
- designation of kinds of working groups didn't reach much of a conclusion

File: content/minutes/design-meeting/2019-11-29-repl-meeting.md

title: Read-Eval-Print-Loop (REPL) Extensions type: docs

Zulip stream or read on the Zulip archive.

Agenda

Design doc: https://hackmd.io/GJokfI0wQ0i4SIgRbFTmfw
Big picture overview and questions
Essential changes to support REPL extensions

Big picture overview of design, and on-the-fly questions about it

Read (to an AST), Compile (AST to MIR), Eval (the MIR to a value), Print (the value). and Loop.
Question: Is this going to be a tool analogous to miri, where check-in's that break it may not break CI immediately, but rather will just file follow-up bugs?
- Answer 1: it has to be a tool; it depends on miri (or a fork of miri)
- Answer 2: it will be even less than a submodule (where a CI automatically files follow-up bugs): There
  won't be any automatic filing of follow-up bugs (alexreg has said
  they will handle maintenance manually); this is how miri and
  clippy had initially started. It could become a submodule later.
Question: Does this mean if we (rustc developers) make breaking
changes to the compiler, we won't necessarily know, right? (Unless
explicit tests are added to rustc test suite.)
Answer 1: Yes, but this is considered acceptable
Answer 2: We can annotate relevant points in rustc with // REPL comments,
(e.g. to avoid them being made private or removed due to lack of use in rustc).
Design doc is split into (interleaved) essential and non-essential changes
For remainder of meeting, we reviewed each essential change at a high level,
and asked participatnts about their thoughts on that change.

Add an “interpreter mode” to the compiler interface

E.g. conditionally prevents dead user variable elimination.
for example, when you do let x = ... in the repl, you want x to be kept around for the future repl inputs.
Question: Are there other contexts where one would the dead-user-variable-elim optimization disable?
Answer 1: Probably not
Answer 2: Patching a running binary, perhaps
Question: What deos it mean to prepend local variables from previous evaluation sessions?
Answer: For each live variable add #[rustc_repl_alive] let var: type;
Conclusion: No one objects to the addition of the "interpreter mode" flag

Slightly expanded miri API

"Added insert_alloc method to machine, used by REPL for restoring memory when deserialising previous evaluation session."
- There was some debate about exactly where this would need to go, but
  everyone agreed that the functionality was necessary.
"Added hooks before_statement, after_statement, before_stack_push (renamed existing method), after_stack_push, before_stack_pop, after_stack_pop (renamed existing method)."
- no objections to these additions
"Made stack pop behaviour more flexible, so as to allow the cleanup flag to be independent of wherever the action is null or a “goto”."
- Question: What are the stack pop changes?
- Answer: it's important that we can exit a block / pop off the stack but still not perform cleanup (so as to be able to serialise the values of the locals when execution is done).
- Question: Is the cleanup the running of destructors?
- Answer: more like clearing the local's memory
- no objections to this addition

Any remainning conerns from stake holders?

We reviewed some vague misgivings with certain design decisions, but
no one had strong objections.

Its not typical to accept changes like this for supporting other projects,
but it is also not unheard of (e.g. miri, priroda, clippy).

Conclusion: people are generally okay with moving forward with what
alexreg laid out, at least the Essential portion

File: content/minutes/design-meeting/2019-12-06-end-to-end-query-PRs.md

title: 2019-12-06 end-to-end query PRs type: docs

Design Meeting 2019-12-06 -- librustc_interface queries

Background

Goal is end-to-end queries
- Benefits: Simpler overall model
- Incremental extending back to parsing
  - mw has done measurements suggesting that on larger crates some 40% of the compilation time comes from
- Parallel compilation would be more effective, if done on the basis of queries
Agenda
- Tough to frame:)
- Most important question to settle is what overall stategy we plan to take
  - Do we want to try and land / rebase these PRs?
  - Try a different tack?
  - Hold off and let other things progess?

Commentary on the meeting

This is written after the fact, and is an attempt by nikomatsakis to capture some of the key considerations.

The PRs were written with the goal of moving rust immediately to an end-to-end query system
- the queries that get created here are not, however, as clean as one might expect
- as an example, instead of creating fine-grained queries for handling HIR, we would create a single HIR Map query and retain today's special case code that tries to track which bits of the data the methods on the HIR map access
An alternative approach would be to move more slowly but work more on the actual design of each piece
- We might start with the HIR, decide on the actual representation that we want, and refactor into that
- And then move backwards to name resolution
- This overlaps somewhat with rust-analyzer, which has been built "from the ground up" with queries in mind, and thus has been figuring out some of what is needed here
For context, there are definitely benefits from changing the line around the "set of things captured in the query system"
- e.g. big parts of the webrender-check compilation takes place before incremental even starts
The concerns with the PRs are that
- we are kind of creating more tech debt before we start to pay it off
  - since the designs are not the designs that we ultimately want
- there isn't really a precise enough consensus around the end state that we want
  - and thus trying to move incrementally means we are kind of ambling without a clear goal
  - and likely to wind up with something incoherent
- the PRs introduce more special cases into the query system, not fewer
On the other hand
- moving quickly to create queries might unlock other improvements, help us eliminate shared state
- maybe you see this as an "obvious first step whatever we do", in which case there isn't a lot of room to block
  - counterargument is that these PRs are each quite complex and take a lot of reviewing load to manage etc
We discussed a fair amount what a desirable design for HIR might look like, starting around here
- it seemed like there was general consensus around a "vague sketch" where you had tcx.hir(def_id) give you back some kind of ItemData that contained the data for a particular item
  - no 'special cases' needed in the query system
Alternative proposal
- close the PRs
- encourage creation of a WG to work out a design for HIR, HIR-ID and the like and bring that design forward
- land new PRs working in that direction
- this overlaps heavily with rust-analyzer and could even take place in that context
Towards the end, some topics were raised that were not fleshed out
- such as whether a DefId could be a "interned" DefPath (or whether it already is)
- the need for 'queries that depend on queries' and what that means

File: content/minutes/design-meeting/2019-12-20-major-change-process.md

title: 2019-12-20 major changes process type: docs

Major Change Process

Zulip thread

Summary

We want some kind of system where people advertise changes that they are making or plan to make
- and the team can give high-level feedback early
- and -- if we decide to go with the change -- we can ensure there is a reviewer beforehand
This document describes motivations and a specific "early draft" proposal

Motivations

Proposal is to add some sort of notification / lightweight process before making major changes.

In contrast, today there is no stated process for a "major change" beyond just opening a PR.
Of course some changes get a lot of discussion before hand, but others do not.

Some problems that we see with today's system:

Sometimes we get large PRs that have attempted a major change without any discussion beforehand
- There may not be a reviewer with time and interest
- Also reviewing without context is very difficult
- There may be concerns about the technical approach or direction
- Starts the discussion with "accept or reject this PR" versus "what is best approach"
  - On the other hand, a more concrete discussion can be more effective
Often, as a result, these PRs can sit for a very long time without any feedback
- this is frustrating for everyone involved
Current "lack of system" can also be a turnoff
- do a lot of work to prep PR, but can you know if that change is welcome?
Over time, a focus on PRs (versus explaining the idea) leads to
- more tech debt and less overall cohesion
- lack of documentation

Some strengths of today's system that we would like to preserve:

Low barrier to entry, not a lot of "bureaucratic overhead"
People should be able to experiment without "authorization" before hand
We don't want a lot of overhead for the compiler team to manage some authorization process

We would know the system is working if:

We have a better idea of what is being done and by whom
- and also whether the team has approved of that direction
Major changes will be discussed before they become a PR that is up for active review
- reviewers will be identified beforehand
When reviewing a PR, reviewers will have a better idea of its goals and how it fits into the bigger picture
We still have a "high throughput" system and we don't spend a ton of time on "bureaucratic overhead"
- in particular we should be able to "green light" changes fairly quickly and we should do that most of the time

Initially proposed process

When considering or experimenting with a "major change", open an issue on compiler-team repo to let people know
- Describe the general idea in a sentence or two
- Identify mentors or reviewers, if you are working with one
These issues will be reviewed weekly and classified
- New triage process? Part of existing process?
  
  [name=Felix S Klock II] Note we already often go-over allotted time at the Thursday triage meeting. We can add this to the agenda, but we need to figure out what else will get de-prioritized.
- Is weekly a good frequency?
  - mark: this is pretty high latency for some of these changes,
    we might otherwise merge them in a week's time
    Maybe that's not a bad thing though :)
Some possible outcomes:
- Closed -- this doesn't seem like something we want to do
- Requires design meeting -- requires a larger group
[name=Felix S Klock II] should also add "Needs RFC" as potential outcome (a more extreme variant on "Needs Design Mtg")
- Deferred -- not deciding for now, or trying to find a reviewer
- Approved for experimentation
  - once PR is ready, nominate for discussion
  - may request "mitigation", such as a -Z flag
  - implementor should understand that we may just decide the idea isn't worth it
- Approved to land -- requires a willing reviewer
  - no special approval required to land, just r+
If a "major change" PR is opened without going through this process
- Close with a friendly note recommending an issue be opened (we should write a standard template with a link)

What is a "major change"?

"You know it when you see it"
If it is a major time commitment to review it, it probably qualifies
- This might be beause it affects many parts of the compiler
- But it might also be a narrow change with subtle implications
- Or require reading up on a relevant RFC or other background material
Examples of major changes:
- Allocate HIR on an arena:
  - Yes, because it touches a set of data structures used throughout the whole compiler
  - However, presuming we could find a reviewer, this would be something we'd like approve quickly within the initial triage because it is fairly mechanical and doesn't require a design meeting
- Introducing chalk, nll, or polonius
  - Yes, replacing a major component
  - This kind of effort might be "redirected" to forming a working group to help lead the design and implementation
    - (which is of course what we have done)
- Changing universally used internal APIs
  - Heads up, PlaceBase::Static is being removed, breaking every single use site of place.base: https://github.com/rust-lang/rust/pull/67000
Grey zone, let's discuss:
- Const propagation
  - It is major in that it is something that has been discussed quite a bit, multiple people might have ideas on it
  - It is low-risk because it can easily be turned off at any time
- Use the recorded types in MIR to determine generator auto-trait implementations
- Implement RFC 2532 – Associated Type Defaults
  - contained, but reviewing is a lot of work
  - Niko has rather inexcusably let this sit far too long (plans to change that)
  - But it's an example of something that's hard to schedule, and where some up-front notice might've been helpful (or could've given warning that won't have available bandwidth until later)
Examples of minor changes:
- Fix some ICE
- Local optimizations

Why this will help

For people who have very full calendars, being able to have a "heads up" of larger changes and to integrate into a review schedule could be helpful
- but it will take discipline to use effectively

Notes from the meeting

simulacrum points out that it would be good to have some kind of "fast path" if you have a reviewer and you have documented things, so that no meeting is required at all
- the only "hard block" would be if you don't have a "partner" or "sponsor" from compiler team
- somewhat analogous to the project group lang team concepts
reviewer not expected to be a pair programmer
one possible definition for "what is a major change" might be "what would modify the rustc-dev-guide"
- or, since rustc-dev-guide is always a WIP, "welp this should've in the guide and if it were, it would've required a modification"
what to do with new PRs that don't follow the process?
- should we close them?
- maybe have a canned comment and give them some amount of time
- this comment might also emphasize the role of documentation
we discussed and settled on
- leave a nice message, which it S-waiting-on-author
- close per usual triage process if no issue is filed within a certain amount of time
- if an issue is filed but it is not "green lighted", then we can close the PR
  - i.e., if we decide that a design meeting or broader project group is needed
when a project is proposed, what are the possible responses?
- I have concerns
- I approve but don't have ability/time to review
- I approve on an experimental basis; we should discuss again when we gain more experience
- I am happy to review but I would like another to approve too
- I am happy to review and I think we can just go forward (only possible for "members")
can we make a Zulip stream where each issue creates a topic?
how to handle experimentation?
- we should have some way to add "caveats", like
  - would like to review performance results
  - we need a -Z flag
  - we need docs :)
final discussion point was about exactly how to handle requests for rustc-dev-guide edits
- since a major change is part of a rustc-dev-guide change, it makes sense that it should come accompanied with a rustc-dev-guide write-up
- ideally this would come along with the compiler-team issue
  - but maybe it would be more something we wait for until issue is approved or, in extreme cases, co-develop with author
- if we want to see more docs, we are going to have to start holding the line somewhere
- sometimes it's not possible or desirable to write complete docs before-hand
  - details may change through review process
  - person may not know enough context to write the docs, need help with that
- but the bar should be that the issue can explain the change in sufficient detail for it to be understood
  - the actual rustc-dev-guide changes themselves can come later
  - it may be that the role of the rustc-dev-guide wg can be to help with some of that

Final proposed process

When considering or experimenting with a "major change", open an issue on compiler-team repo to let people know
- Describe the general idea in a sentence or two
- Identify mentors or reviewers, if you are working with one
- There will be some "prototype" to guide people in this
What is a "major change"?
- something where it would make sense to update rustc-dev-guide
- if rustc-dev-guide doesn't cover this code yet, then you may have to use your imagination about what ought to be documented :)
These issues will be reviewed by compiler team members
- Compiler team members and contributors can leave concerns and approvals asynchronously (see below)
  - maybe we can make a dedicated Zulip stream where new things get
- In particular, note that a compiler team member who is confident something is correct and will not be controversial can just go ahead and approve and act as reviewer
  - though there should still be an issue
- But there should also be some synchronous sweep, not clear when that should occur
  - maybe as part of meta wg?
  - perhaps just team co-leads do it on a regular basis?
  - existing triage meeting is too full, that's clear
Feedback from a compiler team member or contributor typically has the form (these are not fully orthogonal):
- I have concerns (give details)
  - this might lead to more details
  - or a design meeting
  - or an RFC
  - or just closing the idea
- I approve but don't have ability/time to review
- I approve but with some caveats, e.g. we should examine perf afterwards, would like to re-review, or want a -Z flag
- I am happy to review but I would like someone else to approve too (must be a "compiler team contributor")
- I am happy to review and I think we can just go forward (only possible for "members")
  - the idea here is that if you are an expert on the code and confident this is a good path, that's fine, do it
If a "major change" PR is opened without going through this process
- We post a standard comment that directs people to open an issue
- And the PR is marked as waiting on author
  - It can be closed per the usual triage process if author does not respond
- If the issue turns out to be controversial (i.e., nobody steps up as reviewer write away), then we close the PR and just focus on discussing in the issue

A few more last minute notes

Centril + nikomatsakis discussed a bit after and noted that we
probably don't need to get formal about members vs contributors, the
key point is that if somebody is confident that (a) they know this
is the right approach and (b) they know others will agree, they
should feel free to move a major change forward (and leave notes to
that effect on the issue)

File: content/minutes/design-meeting/2020-02-28-focused-and-efficient-triage.md

title: 2020-02-28 focused and efficient triage type: docs

Focused and Efficient Triage

The goal of the meeting was to discuss the idea of creating a
pre-triage working group, the tasks we need to do and who should be
doing them. The motivation is that (a) a lot of work for our current
triage process is falling on @pnkfelix and (b) there is a kind of lack
of clarity around our goals, how we use our labels, etc.

Things we need to do

Monitor and identify "critical bugs" that are not making progress
For critical bugs not making progress, find someone to fix
Making general quality improvements and enhancements (i.e., "fixing non-critical bugs")
Ensuring deferred things are picked up again (e.g., future compatibility warnings)
Processing new issues, ensuring labels are up to date, identifying bugs that are out of date or have been fixed

Typical flow for an issue

Issue gets filed
Release team applies labels
- area labels, team labels
If bisection, mcve needed:
- tag with needs-bisection, needs-mcve
- cc "Cleanup crew"
In some cases:
- directly prioritize or send to the right place if it seems clear
- otherwise, nominate for compiler team meeting
Release team nominates for compiler team to further process
- also cc folks to bisect
- (usually ICE), usually includes needs-bisection and needs-mcve
Compiler team triage group analyzes and figures out which things apply
- Critical bugs:
  - Tag with P-critical
- Needs team discussion:
- Delegation:
  - Should we cc

Proposal

We should have two distinct triage groups compiler and release
- T-release/triage does try to "classify" bugs before it goes to t-compiler
- T-compiler/triage wg should be deciding if bugs are critical (potential release blocker) or not
We would need to tag critical bugs P-critical
- That's a key area of focus
- Those are potential release blockers

T-compiler/triage WG scope

Process I-nominated issues and process those into categories, critical, needs team discussion, delegate)
- The main focus should be to identify new P-critical bugs.
Revisit the set of all P-critical bugs to double check that they have logged progress.
Prepare key highlights to guide the main weekly meeting

Charter for T-compiler/triage WG

Processing 'nominations' and routing bugs to folks who can fix them
Identifying critical bugs and monitoring them to ensure they are making progress
Identifying the agenda for compiler team triage meetings
- Critical issues that are not making progress
- Issues where bugs are nominated for needing wider discussion
  - Ideally, crystallize
Tracking deferred things and ensuring they are picked up again
- Future compatibility warnings

File: content/minutes/design-meeting/2020-03-12-shared-library-for-types.md

title: 2020-03-12 shared library to represent Rust types type: docs

Shared library to represent Rust types

Notes from the meeting

The later sections of this document were largely prepared before the discussion.
This section contains notes from the discussion itself.
You can also read the complete discussion in our zulip-archive.

Conclusions

consensus from folks in the meeting:
- are "cautiously optimistic" about this approach, and open to us trying to create PRs that move rustc in this direction
- are ok with having a rusc-ty library that is published to crates.io
but there remains disagreement about the big picture question of how stable API boundaries can/should be between libraries, whether they should live in their own repos, and whether they should use unstable features (but that was not the direct topic of this meeting, so that's ok)
sketch of a plan for how to do the transition:
1. ty.kind -> ty.kind()
2. ty.kind() -> ty.kind(tcx)
3. Add a type alias and make kind work on I: Interner
4. Start using Ty more (This is a lot of PRs)

Key or interesting points

Some key or unique points that were raised during the discussion:

abstracting more over the concrete representation of types permits experimentation for performance reasons
impact of more generics:
- more generics may increase compilation time, although this is not clear, and we also get the advantage that we can write libraries that do not depend on most of rustc. They would use traits like the proposed Intrner trait or chalk's RustIrDatabase trait to isolate out their dependencies.
- we won't be able to use pattern matching
- generics also permit logically separating out different sets of types in your code (e.g., inputs and outputs), even if they both ultimately share the same representation
interactions with MIR
- types include constants, which reference MIR, is that a problem?
  - no, because types reference MIR through def-ids
- rustc-ty feels like a dependency that could be used to build a rustc-mir that just defines MIR (but independently from rustc)
to do the transition:
- two main parts that are somewhat independent:
  - reshape TyKind to a more chalk-like hierarchy
  - introduce the interner, which can be done via a series of PRs like ty.kind -> ty.kind() -> ty.kind(tcx), and as described in the proposal below
- one question was whether we should we do more of the experimentation in rust-analyzer or chalk first; but doing the work on rustc means we will encounter the interesting side-cases. Still, we have done the work on chalk already and should start migrating rust-anayzer to share representation with chalk, too

Deferred questions

Some bikeshed or other detailed questions were raised but largely deferred:

rustc today uses ty.kind to get the "kind" of type, chalk uses ty.data() -- which name is preferred?
we may want to support flags also on "lists of types"
what to name the crate
should the library use unstable features and how stable should it be?
- this was largely deferred but we did get into some of the interesting details, particularly on the question of stability
- being more stable (even just noting when there are semver-relevant changes) requires us to take on some more overhead
  - maintaing version numbers, maybe a changelog
- avoiding nightly means we can't dogfood features, some of which don't see that much usage in the ecosystem
- but:
  - it would be great it rustc development felt much like working on any other rust project, e.g. using cargo + stable
  - and having libraries that people can build on would eventually enable more complex tooling like Roslyn; people want this, think Mirai
  - if we can find meaningful boundaries between crates, maintaing semver shouldn't be so hard (but can we?)
- ultimately we deferred the question to discuss more explicitly and with more experience

Context

As part of library-ification, we need a library to represent Rust types. The intent is that this library can be used by chalk, rustc, rust-analyzer, and any other projects which need to represent Rust types.

Goals:

Do not hard-code the use of interning. Interning is a good fit for batch compilation but not such a good fit for IDEs, which need to be able to gradually release memory on an ongoing basis.
Ergonomic library that is pleasant to work with.
Simplify Rust's types to "their essence", while retaining full information

Goals for this meeting

Discuss the "high-level plan" for the design
Discuss the roadmap and initial steps to work towards this goal

Non-goals

We do not need to decide the fine details of the type (e.g., what variants there are, what to name the methods, etc) at this time. We will work those out through PRs and refactorings.
Figure out the mono-repo vs poly-repo question and where the shared crate should ultimately live (I sketch some interim solutions below)

Proposed end-goal: crate structure

This idea is part of a broader 'library-ification' effort. The goal is to build up independent crates for the various parts of rustc, like the trait system, type checker, name resolution, etc, that can be shared by both rustc and rust-analyzer.

This particular meeting is talking about what might be part of a crate that describes Rust types, which we'll just call ty for now. This crate would be the foundation for a lot of other crates:

text

"rust-analyzer"
                               ^
                               |
ty --> "trait system" -----+---+--> "rustc"
 |          ^              |
 |          |              |
 +---> "type checker" -----+

In other words, we are talking about ultimately replacing the Ty<'tcx> type that rustc uses today with a type defined in this ty crate, modeled roughly after chalk's Ty type.

Proposed end-goal: the basic pattern for representing types

The proposed design is described in some detail in the chalk book, so you may wish to read there. However, I do want to emphasize that the fine details (e.g., exactly which variants we should have) should not be considered normative.

In the proposed design, Rust types will ultimately be represented by a struct that looks like this, based on chalk_ir::Ty:

rust

pub struct Ty<I: Interner> {
    interned: I::InternedType
}

impl<I: Interner> Ty<I> { ... }

Clearly, Ty itself says very little. It is just a "shell" that wraps around the interned representation, which is defined via the Interner trait:

rust

trait Interner {
    type InternedType;
    
    fn intern_ty(&self, data: TyData<Self>, flags: TyFlags) -> Self::InternedType;
    fn lookup_ty_data<'ty>(&self, ty: &'ty Self::InternedType) -> &'ty TyData<Self>;
    fn lookup_ty_flags<'ty>(&self, ty: &'ty Self::InternedType) -> TyFlags;
}

To actually use a type, you wish to invoke the data() method (you have to supply a reference to the interner):

rust

impl<I: Interner> Ty<I> {
    pub fn data(&self, interner: &I) -> &TyData<I> {
        interner.lookup_ty_data(&self.interned)
    }

    pub fn flags(&self, interner: &I) -> &TyFlags<I> {
        interner.lookup_ty_flags(&self.interned)
    }
}

This method returns a reference to a TyData, which is the actual enum that defines the type variants. The definition that follows is taken from Chalk:

rust

enum TyData<I: Interner> {
    Apply(ApplicationTy<I>),
    Placeholder(PlaceholderIndex),
    Dyn(DynTy<I>),
    Alias(AliasTy<I>),
    Function(Fn<I>),
    BoundVar(usize),
    InferenceVar(InferenceVar),
}

As noted before, don't focus too closely on the precise set of variants. Chalk aimed for a very minimal set, and the idea was to justify each variant by explaining clearly why some bit of code might want to look for types of that variant, but I expect that we'll ultimately want to find some "sweet spot" in between rustc's approach (lots of variants) and chalk's approach (minimal set).

The key thing to discuss here:

A Ty type with an I: Interner parameter that defines its representation

Interner in Rustc

The interner in Rustc would be TyCtxt<'tcx>, and hence we'd have something like

rust

impl<'tcx> Interner for TyCtxt<'tcx> {
  type InternedType = &'tcx TyData<Self>;
  
  fn intern_ty(&self, data: TyData<Self>) -> &'tcx TyData<Self> {
      self.interners.allocate(data)
  }
}

How to get there from here

Presently, rustc, chalk, and rust-analyzer each have their own type definitions. The goal is eventually that all three projects should share the same crate that defines the type definitions, using the above generic style. But how do we get to that goal? And where should that crate live? This ultimately plays into the mono- vs poly-repo discussion and I would rather not go there in this meeting. It seems like we can address the questions a bit later when we have made more progress.

Until then, I propose that we gradually refactor rustc so that the types in rustc approach the goal described above and ultimately factor out a rustc-ty crate that contains just the type definitions (and some supporting types). This should be a crate that can be published on crates.io (as well as living in tree). chalk can then depend on this crate. Once that is done, we could consider whether the crate should move to chalk (which would be moving towards polyrepo) or whether it should remain in tree, or whether we adopt some other, hybrid solution (as discussed here).

In more detail:

Rework the TyS API so that you access the variants by ty.data(tcx) and not ty.kind (similarly flags)
Introduce an Interner trait into rustc and refactor from type Ty<'tcx> = &'tcx TyS<'tcx> to type Ty<'tcx> = ty::Ty<TyCtxt<'tcx>>'
Extract the type stuff into a rustc-ty crate
- this should have no dependencies on other parts of rustc and be something we can publish to crates.io
Make chalk depend on that crate

chalk integration

Note that while the above work is proceeding, we may also want to experiment with "chalk integration". This would be done by having -Zchalk cause rustc to translate its types (however they may be represented) into the minimal chalk-ir types on demand. As rustc's types come to match chalk-ir types (and/or chalk-ir types change to match rustc types), this would be a simpler and simpler translation and would eventually become a no-op once rustc + chalk share the same type definitions.

Where should this crate live?

This crate will start to re-raise the question of poly- vs mon-repo. I would like to dodge that particular discussion for now. What I've proposed above has a rustc-ty crate living in tree. This is obviously the right place to start, but ultimately I think we should consider the idea of having "type related code" living in a separate rust-lang repo (like rust-lang/chalk), which might ultimately include the representation of types, the trait solving code, and perhaps the type checker (and depend on some other repo which provides the HIR that gets type-checked). But obviously this would carry coordination costs and all of this seems far enough way I'd rather not discuss it in this meeting.

File: content/minutes/design-meeting/2020-04-03-cranelift-backend-for-rustc.md

title: 2020-04-03 Cranelift backend for rustc type: docs

Cranelift backend for rustc

You can also read the complete discussion in our zulip-archive.

Meeting Proposal

Pre-meeting details and agenda outline

Overview of the Cranelift backend

Cranelift is a code generation backend written in Rust which is under development for use in WebAssembly compilers.
As such, it could potentially improve compilation times in rustc because it is optimized for efficiently generating code rather than generating efficient code.

Over the past ~1.5 years, @bjorn3 has been working on writing a Cranelift-based code generation backend for rustc: rustc_codegen_cranelift or cg_cranelift for short.
It is currently complete enough to compile many programs and is usually faster at compiling them than the regular LLVM backend.

The main goal of this project is to build an alternative codegen backend that can be used for faster, non-optimized (debug) builds.

rustc has support for runtime-pluggable backends via the unstable -Zcodegen-backend=/path/to/dylib flag.
cg_cranelift is developed in a separate repository from the Rust project, so it uses this flag to plug into rustc.
cg_cranelift also takes advantage of an internal rustc library called rustc_codegen_ssa which tries to be a backend-agnostic code generation support library but some parts of it are still very LLVM-like.

Next steps

At this time, we want to help increase the support cg_cranelift gets but it isn't a top priority for us currently.
The eventual goal seems to be first-class support for cg_cranelift alongside our existing LLVM backend but that won't happen for a while so we can afford to take small, measured steps in the meantime.

A few things we would like to do now:

Continue refactoring rustc_codegen_ssa to be less LLVM-like and allow cg_cranelift to re-use more of the code.
Bring in cg_cranelift to the rust repo (probably via git-subtree) and require it to build during CI.
- This is similar to the support we give to tools like clippy or miri.
Add a way in our testing infrastructure to mark tests as reliant on a specific codegen backend and annotate existing tests that require LLVM.
Add a way to manually run tests with cg_cranelift to Rustbuild (aka x.py).

We may also want to do these soon but we need additional discussion:

Run tests in cg_cranelift mode and notify if they failed (but don't gate CI on it).
Form a working group related to cg_cranelift.

File: content/minutes/design-meeting/2020-05-29-Roadmap-2020-2021.md

title: 2020-05-29 Roadmap 2020-2021 type: docs

Key notes and points from the meeting itself

Parallel compilation:
- There was a general consensus to deprioritize parallel compilation
  in favor of pursuing better improvements to incremental compilation.
- One interesting use case is CI turnaround times. Parallel
  compilation doesn't do much there since most CI systems are not
  very beefy. Incremental doesn't either, but it might if you
  enabled incremental support (and could trust it well enough).
- We can still land some small PRs etc without it being on the
  roadmap necessarily.
Incremental compilation:
- pnkfelix would like to get an incremental working group up and going.
- simulacrum is interested in working on improving profiling.
impl Trait: we discussed a bit what it would take to finish up impl Trait.
How to drum up excitement:
- WesleyWiser raised up the 2018 impl period as a really motivating
  and clear way to get involved:
  - "I think one of the things I really liked about the 2018 impl
    period was that it created a really smooth ramp to get people
    involved with making the project goals
    happen."
  - A lot of the people who joined them, it seems, stuck around.
- We discussed some of how we might recapture a bit of that spirit, with the thought
  that a big part of it
  - breaking down into a list of groups with goals
  - a marketing push advertising those rgoups and projects
  - identifying people who are happy to taker questons etc
We discussed "saying no" to some of the things on the list, notably
a lot of them are assigned to nikomatsakis.

Initial proposal

This section contains the proposal that was made up at the start of the meeting.

General areas and themes

These are not meant to be specific projects, but more categories that justify why we are pursuing specific projects. The categories include a bit of "what" that clarifies what the category is as well as "why" indicating why this is a category of work worth pursuing.

Improving the IDE experience
- Why? In survey, a strong IDE experience was one of the top reasons given for "why I don't use Rust" or "what would make me use Rust more often", and the largest one that is directly under compiler team's control.
Improving build times for CI
- What? From-scratch batch compilation performance
- What? Integration with
- Why? Anecdotal feedback that "interactive execution is ok, but CI is painful"
- Why? Persistent complaint when it comes to large codebases especially
Polish and quality assurance
- What? Fixing bugs, improving diagnostics
Deliver on long-standing features, finish things up
- Why? Rust 2020 roadmap goal
- Why? Anecdotally, a sense of "everything I want to use is always half-finished" has come up from numerous surveys (e.g., compiler team survey, a mozilla internal surveys) as well as frequent conversations.
Async polish and performance
- What? Improving diagnostics, performance of async-await
- Why? In survey, servers are the number one application for Rust.
Improving the way we organize ourselves
- What? Structural changes to help us engage newcomers and be clearer in our priorities.
- Why? Acting on the feedback from compiler-team survey that we need more prioritization and ability to communicate what we're doing.
- Why? Help address the need
Improving incremental compilation
- What? Making more source inputs incremental
- Why? Improve local edit-compile-debug cycle
- What? Increase test surface (e.g. discover new cases to test)
- Why? Most incr-comp bugs reported do not have enough info for replication.
  - (An alternative here: Figure out ways to capture needed info automatically)

Top goals for 2021 (Edition)

These are actual specific projects. The idea here is to be thinking about "things that would be complete by mid 2021" -- but really I have in mind "what can we get done by 2020", per Hofstadter's Law. The list is perhaps too ambitious. It's not meant to be exhaustive necessarily, but it should I think try to identify the major things we think we can realistic accomplish and is meant to help us in driving our efforts. In particular, places where a single person is driving multiple projects are somewhat suspect and we should look carefully at which is the most important thing. It's better to do one thing completely than two things half-way, in other words.

For each goal, we list the following:

Motivation: one of the motivations above
Driver(s): people who are driving this work, if known (typically leads of groups)
Group(s): groups driving this work, if any
Goals for 2021: try to enumerate the thing we think we can accomplish in more detail

Goals

Triage work
- Motivation: Polish and quality assurance, Improving the way we organize ourselves
- Driver(s): pnkfelix, spastorino
- Group(s): wg-prioritization
- Goals for 2021:
  - keep on keepin' on?
  - Keep the weekly triage meeting focused and on-time?
Rust-analyzer transition
- Motivation: Improving the IDE experience
- Driver(s): matklad, flodiebold
- Groups(s): wg-rls
- Goals for 2021:
  - moving rust-analyzer to be default IDE solution
rustc-dev-guide
- Motivation: Improving the way we organize ourselves
- Driver(s): spastorino
- Group(s): wg-rustc-dev-guide
- Goals for 2021:
  - Reorganized table of contents
  - Complete walk through?
  - Slowly fill out gaps in useful knowledge
Progress towards chalk integration
- Motivation: Improving the IDE experience
- Driver(s): nikomatsakis, flodiebold, jackh726
- Group(s): wg-rls, wg-traits
- Goal for 2021:
  - rust-analyzer is using chalk, enable it to type-check and manage most projects
  - extract a shared library for types usable by rustc, chalk, and rust-analyzer
  - integrate experimentally with rustc and get to the point where -Zchalk allows things to largely work
MIR Optimizations
- Driver(s): oli, wesleywiser
- Group(s): wg-mir-opt
- Goal for 2021:
  - Speed up compile-time by removing work llvm has to do
  - Speed up runtime by doing optimizations that llvm can't do or doesn't have the (type?) information in order to do.
    - Basically make idiomatic code as fast as handcrafted
Generic associated types
- Driver(s): nikomatsakis, matthewjasper perhaps
- Group(s): wg-traits?
- Goal for 2021:
  - Have some form of GATs, including type-generic associated types, usable in rustc under a feature flag
  - Fix known soundness bugs
Const generics
- Motivation: Deliver on long-standing features, finish things up
- Driver(s): varkor? who else?
- Group(s): XXX no project group at present
- Why this feature?
  - Commonly cited in the Rust survey as one of the things people are most excited about
  - Blocks a number of language improvements, such as SIMD, let x: [u32; _] = ..., etc
- Goal for 2021:
  - Correct the bug around the "generic parent" (blocked on lazy normalization)
  - Stabilize? Is that too ambitious? Can we clarify other goals?
Const evaluation
- Motivation: Deliver on long-standing features, finish things up
- Driver(s): oli, ralf, ecstatic-morse
- Group(s): wg-const-eval, miri
- Why this feature?
  - Commonly requested feature for embedded (very important there) and for replacing lazy_static things with compile-time things
  - Loads of finished features that need a bit of carrying over the stabilization line.
Improve the performance of the trait system
- Driver(s): eddyb, to a lesser extent nikomatsakis
- Goal:
  - Complete work described in w3f#261
Introduce major change proposals (MCP)
- Motivation: Improving the way we work
- Driver(s): nikomatsakis
- Group(s): wg-meta
- Goal for 2021:
  - See widespread use of MCPs for major changes
Introduce groups for major areas in the compiler as well as targets
- Motivation: Improving the way we work
- Driver(s): nikomatsakis, spastorino
- Groups(s): wg-meta
- Goal:
  - Create clearer folks that
RFC 2229 Implementation
- Motivation: Deliver on long-standing features, finish things up
- Driver(s): nikomatsakis, matthewjasper
- Group(s): wg-rfc-2229
- Goal:
  - Land an initial version of the feature
Create a wg-incr-comp to focus on improvements to incremental compilation
- Motivation:
- Driver(s): pnkfelix, XXX?
- Group(s) wg-incr-comp
- Goal:
Improving profiling infrastructure and self-profile
- Motivation: Enabling other groups to establish metric-driven goals
- Driver(s): simulacrum
- Goals for 2021:
  - Be able to track disk space utilization of incremental, normal compilation
  - Track codegen unit splitting
impl Trait plan
- Motivation: Deliver on long-standing features, finish things up
- Driver(s): nikomatsakis, matthewjasper, oli-obk
- Group(s): wg-impl-trait?
- Goal:
  - Have to define the timeline, but basically clean up and deliver on impl Trait in various positions
  - Coordinate with lang team to see how much consensus we can get on other niggling questions
async-await polish
- Motivation: Async polish and performance
- Driver(s): tmandry, ...?
- Group(s): wg-async-await
- Goals for 2021:
  - Hard to define concretely, maybe we can?

Awesome accomplishments

I'd like to also note some of the awesome accomplishments we've had since Rust 2018. Let's list them out here:

self-profile and its integration into the perf page
Cross-lang ThinLTO and PGO (when did this ship?)
rustc-dev-guide has grown enormously
Const evaluation has made a ton of progress
MCP process introduced :tada:

Projects not currently planned to move forward and why

This is based on the discussion in the meeting.

Parallelization
- Motivation: improving build times, but it's not clear which project above this helps (maybe CI)
- Would need a good driver and plan
- But the code does exist and there are real wins to be had here for people's experience, so quite possibly worth pursuing
- Status for 2021:
  - In stasis
  - Might be willing to pursue, but it would require a real assessment of the plan and a set of committed people

File: content/minutes/design-meeting/2020-09-18-mcp-retrospective.md

title: 2020-09-18 Major Change Proposal Retrospective type: docs

Key notes and points from the meeting itself

Feedback on MCPs after being seconded can be difficult to deal with. (link)
- Some feelings of "where were you a week ago" as aluded to above
- We do want to encourage feedback even if it's late but we should weigh some kinds of feedback more heavily than others "late" in the process.
Most common feedback was: (link)
- the problem of "MCP limbo", where there are MCPs that are not accepted but not clearly rejected
- the problem of how we know if a concern is present and when it's "resolved" (who decides?)
- the question of what to do about feedback after an MCP is accepted, and in particular MCPs for sweeping changes
We should probably be closing un-seconded MCPs after some amount of time (link)
- There should probably be a way for a contributor to keep an un-seconded MCP "alive". Possibly by leaving a comment to that effect.
- That would extend the time before the MCP is closed but not indefinetly. Contributors would have to keep leaving comments if they want the MCP left open (this "friction" is probably a good thing).
- @pnkfelix points out there isn't a good way to track these ideas after an MCP is closed.
- Meeting participants generally seem ok with that.
- There is "fragile consensus" for auto-closing after a set amount of time has passed.
Discussion about "it's hard to track concerns" (link)
- "Obvious" fix is to use rfcbot concerns
- rfcbot concerns may not scale to such a large group as compiler-team + contributors (link)
- A possible solution would be a small group of people whose responsibility would be to determine if concerns have been resolved sufficiently for the MCP to proceed. This likely overlaps a lot with the Technical Steering Committee idea which has been talked about before. (link)
- Discussion of specific rustbot commands (link)
  - rustbot second -- marks this for approval after time elapses
  - rustbot close -- marks this for close after time elapses, unless a contributor leaves a comment
  - rustbot keepalive -- cancels a close
  - rustbot concern foo -- raises a concern, any team member/contributor can do it? (or anyone?)
  - rustbot resolve foo -- resolves a concern, any team member/contributor can do it
  - if an issue goes N weeks without activity (on the github thread), there is an automatic rustbot close
Discussion about MCPs that perhaps don't get enough attention (link)
- The biggest example of this is probably the recent MCPs that changed the rust-lang/rust repo directory structure and the x.py defaults. (link)
- Some people didn't find out until after the fact and were unhappy with the approved changes.
- For "large" or "controversial" MCPs, we may want to require a full FCP instead of simply a second. (link)
  - @simulacrum notes if there are conflicting veto/second comments, perhaps this should trigger a full FCP either immediately or within a certain amount of time if those concerns are not resolved. (link) (link)
@nikomatsakis noted that the feedback from the survey showed nearly everyone agrees the MCP process is good for the team. (link)

File: content/minutes/design-meeting/_index.md

title: Design meeting type: docs

Design Meetings

File: content/minutes/steering-meeting/2018-10-26.md

title: 2018-10-26 type: docs

2018-10-26

We had our first T-compiler steering meeting today! What follows is a summary of the major points raised during the discussion.
If you'd like to read the detailed minutes, please see the Zulip chat log. I'll insert links here and there into that log where appropriate, if you want to see the original comment.

To start, before the meeting we did a survey asking folks what they thought worked well and what needed improvement in and around with the compiler team. I summarized those results in a Google doc, which shows the things people wrote and how often (in the case of duplicates). We then voted on what to talk about (that's what the columns with the x are).

In the end, we opted to discuss the most popular topic, which sort of subsumed a number of entries:

"how to plan what to do and expose our plans"
- hard to answer "what to do next" or "how can I help" (8)
- public roadmap (4)
- hard to do high-level design + planning and not just triage (3)
- no central place to get "big picture" of what's going on (3)

(Originally, we thought we'd cover many more things, but that did not turn out to be true. =)

To start, there were a number of comments on the survey (and in the meeting) to support the idea that Working Groups are an effective way to organize, particularly for newcomers, but it works best if we focus the WG on a concrete topic (e.g., NLL). WG leaders ought to be actively triaging and looking for places to help people get involved, and so they can be good ones to answer the question of "what to do next".

We can also use WG as an organizational tool to expose the "big picture" of what's going on. In addition to WGs, though, it would be good to just list out the specialities of compiler team contributors, as well. WGs can also be an organizational axis for labels with the WG focusing creating a steady supply of E-mentor issues.

However, this places a big burden on WG leaders. One thing we noted is that having more than one leader is good and that this itself can be a mentoring opportunity.

In terms of getting people involved, the findwork site was raised, as well as the difficulty of keeping it up to date (perhaps WGs can help?).

We pointed out that when we do our planning at the next Rust All Hands, we can try to figure out WGs then.

We discussed some how the diagnostics efforts could be increased.

We discussed the rustc-dev-guide as well as the NLL contributor YouTube videos and ways to improve them. One solid thought was that we might want to create a rustc-dev-guide WG (which has its own zulip topic here). We talked in particular about how both experienced folks and newcomers can contribute there.

File: content/minutes/steering-meeting/2018-11-16.md

title: 2018.11.16 type: docs

2018.11.16

The topic of today's steering meeting was future
planning. Specifically,
we were looking at some of the ideas we've had for major initiatives
and trying to drill into them and make them more concrete. You can
read the full thread over on
Zulip.

We began by observing that the ultimate decisions we make here will be very influenced by the number of leaders we can find, and what they want to do, and ultimately decided to focus first on the technical angle before thinking about what leaders we have etc.

We made an initial breakdown of the work to be done into some big areas, which was later amended with a few more areas:

parallelize with many threads
improving "batch compilation":
- do what we do now, but better:
  - type-check faster, trait checking, etc
  - optimize MIR and generate better LLVM
  - do things better in the backend / improve ThinLTO (link)
  - avoid things like dsymutil
  - need to look at "time breakdown" across projects a bit perhaps?
- defer, coallesce, and avoid work
  - multi-crate compilation model
  - MIR-only rlibs, which helps identify duplicates as well as dead code
  - "polymorphization" to coallesce monomorphized instances that have the same code
- distribute and cache work across machines
improving RLS experience:
- end-to-end queries
- incrementalization
- ...
"UX" improvements
- continue compiling even when errors occur, particularly getting through to the borrow check phase (link)
- error-message work
- ...

For the rest of this conversation, though, we focused on batch compilation. We discussed opportunities for getting estimates of the effectiveness of some of these ideas:

for MIR-only rlibs, mw already did some of that work. There is great potential, but we need effective parallelization.
thanks to hard work by Zoxc, mw, and others, we have a prototype parallel implementation, so mw volunteered to gather some data
for polymorphization, we can of course use LLVM's mergefunc to get some idea, but it seems likely that we can find more opportunities than LLVM does, and of course we would also operate earlier so the impact is larger. We would begin by doing an analysis over MIR -- this would be a great place for someone who wants to work on a more weighty project to get involved.
- one concern is that debuginfo and polymorphization may be tricky to combine
in principle we should be able to see, for large projects, how much code we wind up using in the end from dependencies, which might give us wins if we can defer that work through multi-crate compilation

One question mark that was raised was how important it is to extend the query system further into the front-end of the compiler -- that code is currently usually pretty fast relative to compilation as a whole. However, if we want to really transform the RLS experience, we clearly need great progress here. So to some extent this may come down to the relative importance of RLS vs batch compilation. Ultimately, the consensus was that we need to measure and get a clear idea where we are spending our time (shocker).

A lot of these topics we've discussed above will require major architectural overhauls to the compiler, so one thing we discussed was the desire to sketch out those refactorings and the dependencies between them into a kind of chart. We also talked about the need to "pay down" technical debt through cleanup refactorings, and how those refactorings should be given high priority. Finally, we talked some about how the rustc crate graph could stand for some reorganization -- this perhaps falls under technical debt. =)

(Fun fact: we initially split rustc into crates not for compilation time but rather because -- shortly before 1.0 -- the memory usage for compiling and linking the rustc crate was exceeding 4GB, preventing us from issuing builds on 32 bit machines.)

Action items

We enumerated some goals to try and get done before the next steering meeting. If you'd like to help with one of these tasks, let nikomatsakis know!

take another stab at collecting and organizing ideas, incorporating ideas around UX in particular, but also breaking out RLS from batch compilation (nikomatsakis)
gather statistics about where we spend our time from perf and perhaps elsewhere (?)
draw up a proposal for polymorphization analysis / strategy (maybe nikomatsakis?)
collecting some numbers on how parallel queries performs these days (mw)
draw the graph of refactor dependencies (eddyb/nikomatsakis, perhaps?)

File: content/minutes/steering-meeting/2018-12-14.md

title: 2018.12.14 type: docs

2018.12.14

The steering meeting was focused on mentoring. Some of the prompting questions:

How can we help move people from "working group participant" to "full-fledged team member"?
How can we reduce the "bus factor" for cricial modules like the type checker, type resolver, etc?

A summary of the major points raised follows. If you prefer, you can
read the full transcript on Zulip.

Major points: Mentoring

it seems good to try and list out the parts of the compiler and who
is an expert in each
one,
perhaps in some kind of "introduction to the compiler team" page on the repository
- this can help identify where to direct your questions about code X
- but also where we need to increase the number of knowledgable people
it would be great if we had an organized off-ramp system
for folks who have been active in a working group and are looking to broaden their knowledge
- maybe we can pair up such folks with someone
  more
  experienced
- this is a different sort of mentoring, since active people from a
  WG are already familiar with how the compiler works, and thus able to work at a higher level
- we need to find a way to make it easier to find "intermediate-level mentoring opportunities".
we might try to actively increase bus factor via fixing issues, writing tests, or writing docs
- writing tests can be driven via code coverage measurements
- writing docs can be driven via
  deny(missing_docs)
  as well as push to complete the missing rustc-dev-guide
  sections
- one problem with writing documentation is that there always seems
  to be something that is higher
  priority,
  maybe implying we need it as a roadmap item
a dedicated #t-compiler/help
stream
would be a good way to call attention and request mentoring
- maybe other streams as needed
- also, documentation on how to use zulip seems like a win

Major points: process for refactorings, documentation

Towards the end, we also discussed the procedure we should use for
larger compiler refactorings as well as how to ensure the compiler
is documented.

before landing a massive refactoring or new architecture:
- ideally, talk out the design on an internals thread to achieve some light consensus
- expect documentation in the form of rustc-dev-guide chapters
some examples where we should do this:
- end-to-end queries
- parallel query execution
in terms of the rustc-dev-guide:
- it'd be great to improve mdbook to permit for graphs
  - maybe we can add graphviz support into mdbook
- we should not neglect doc comments
  - and we can link back and forth copiously (and make sure we check for dead links)

File: content/minutes/steering-meeting/2019-01-04.md

title: 2019.01.04 type: docs

2019.01.04

We discussed the upcoming Rust All Hands. We have a HackMd
document that lists a bunch of ideas for what to talk about,
but mostly we talked about how to structure the meeting itself.

We summarized how the All Hands was structured last
year:
basically meetings focused around particular topics, with the team
(and others) mostly in a kind of semi-circle or around a whiteboard.

One of the ideas we had for this year is that we could have some
sessions that are just focused on explaining how a particular area of
the code
works,
which could help people unfamiliar with an area get up to
speed.

In general, we decided that each of our slots should begin with some
exposition of "how things work
now",
and then segway into "here are problems to resolve". We will try to
take good notes on these expositions as well.

We discussed how it might also be useful to talk about some of the tools
that are "adjacent" to the
compiler,
e.g., rustdoc or RLS.

We discussed some possibler areas for discussion:

the rustc-dev-guide
related work, like RLS, rustdoc,
or cranelift
mentoring?
- maybe, but we have to figure out specifically what
salsa, polonius, chalk, etc
diagnostics

On the topic of IDEs, it would be useful to hear from Matklad about
rust-analyzer.

We also thought a bit about some more "radical" ideas that might be worth
making a bit more concrete:

File: content/minutes/steering-meeting/2019-01-17.md

title: 2019.01.17 type: docs

2019.01.17

We discussed the upcoming Rust All Hands.

One of the first things we focused on is trying to get consensus
about what the P1 Problems are that we are aiming to
tackle.
Our goal is to have this list complete before the all
hands.
Our initial
list
looked like this:

compilation time
- better compilation time investigation tools/analysis
- this should work in cooperation with cargo, so we can see whole story
rls, completions
“too hard to do anything” — technical debt
“too hard to find people to do things” — organizational debt
- hard to learn, monolithic architecture
- poorly documented
- long compilation times, memory requirements

Looking at it, we realized a few things:

in past meetings, we've discussed how it feels hard to justify time spent writing docs. In general, we felt we needed some form of concrete incentive, similar to how performance optimization gets you that nice green bar. To that end we talked about:
- maybe we can set some specific documentation targets? For example, enabling deny(missing_docs) by summer.
- or identify some other form of doc coverage metric
- we might also block PRs that lower the doc threshold
in general, this seemed like a good topic to revist under a "compiler team governance and apprenticeship" meeting at all hands

In terms of the all hands, one idea for a possible outcome is to try
and produce a "slew of technical design
RFCs",
at least for certain key areas (e.g., the so-called MIR 2.0). We later
softened this to a "technical outline plus a plan for who writes the
full
text",
where a "plan" includes who will do
what.

One possibility is having "breakout
sessions"
where a smaller set of people go off to focus on some questions and
then bring the results back to the group. It's not clear how or if
this will fit in the schedule.

We then turned to the topic of the RLS. Initially, we were
contemplating starting out with some "review sessions", but we
ultimately decided to try and produce that material before the all
hands. Ideally,
for all of our major meetings, we'd have some kind of review material
to get people in sync. Offline material can be enjoyed by
others,
and it's nice for people to be able to review it at their
liesure.

When it comes to the RLS, we decided to take a "tech first"
approach
-- basically focus on the technical design we want, and then talk
about the best way to get it (e.g., improve RLS in place, start over a
la rust-analyzer, etc). matklad, xanework, and nikomatsakis held a
separate discussion to settle on an initial agenda
(topic).

For the "second most important topic", we settled on "organizational
questions". The "root goal" of any organizational alternatives would be:

be intentional about what we are doing: make sure we are making progress on the problems we think are most important
grow the team
maintain/increase quality

Some sample agenda items were how to handle factoring of compiler
into libraries and how it relates to team
structure. There
was a consensus that having more focused subteams would reflect the
reality that the compiler is too complex for one human to fully
understand
(unless they are eddyb).

Concluding comment:

I feel like we made some good progress on the concept of how to approach IDE discusson.:

We will start by elaborating technical design and let that guide us the rest of the way.

We will have some review materials available before the all hands

We will have a short meeting (like, nowish) to try and create agenda

it seems like we consider organizational matters the next most
important thing to discuss as a group. I feel a bit less clear on
what the "agenda" will be, but maybe we'll feel our way, or else
maybe we should have some more pre-discussion about it. I would at
least be up for trying to prepare a kind of straw-person proposal.

File: content/minutes/steering-meeting/2019-03-01.md

title: 2019.03.01 type: docs

2019.03.01

We plan to flesh out some more details and put this idea into action
starting May 3rd (the next steering meeting).

Before the meeting started, @nikomatsakis proposed an agenda and linked to a
document to collect agenda ideas for future meetings.

Also, @mw asked about discussing the policy for out-of-tree compiler crates (expanding on
discussions from the All Hands' organizational meeting), a draft for which was
already in-progress by @davidtwco.

The meeting commenced with a call for involvement in the RLS 2.0 working group led by
@matklad as sharing expertise between the RLS 2.0 effort and rustc would be helpful.
There is documentation (videos and guides), issues with E-mentor and other resources available
for those interested. @davidtwco, @pnkfelix and @Xanewok
mentioned that they would endeavour to find some time to dig into RLS 2.0.

Next, @nikomatsakis announced an ongoing effort by @spastorino to
collate an expert map about who has expertise in
which areas of the compiler, with the intent of making it easier to find the right people to
ask questions and assign issues/PRs to, and invited everyone to help flesh it out and add
themselves. @nikomatsakis also suggested that this would be good in documenting the
overall structure of the compiler in some ways.

Following this, the main topic for the steering meeting, the out-of-tree crates policy was
discussed (for things like polonius, chalk and the upcoming diagnostics
refactoring). The current draft of the policy was linked and @davidtwco
emphasised that it was very much a draft and there are lots of unresolved questions.

@nikomatsakis highlighted a desire for having a uniform experience between
out-of-tree crates and normal in-tree development - particularly surrounding who can land PRs,
highfive reviewer assignment, and ensuring anyone in the team can publish new crate versions.

@davidtwco noted that discussions surrounding tooling that could make out-of-tree
crates easy to work with was left out of the policy draft as that isn't part of the policy.

Both @nikomatsakis and @mw wanted there to be a simple guide to follow for
creating out-of-tree crates - this should include links to examples and things to copy-and-paste.

@pnkfelix raised concerns about current policy wording suggesting that only
general-purpose functionality should be factored out and suggested clarifications about the minimum
bar for expected re-use might be beneficial. @nikomatsakis mused that making
libraries too general purpose could carry a higher maintainance/breaking-change burden. @mw
suggested that this would need to be decided on a case-by-case basis. @nikomatsakis
clarified that we should absolutely be following semver in these crates.

@nikomatsakis then raised the question of review policy in out-of-tree crates -
should major PRs require multiple reviewers? can crates have reviewers that are not reviewers of
rustc? Going to on say that the idea of these crates having their own communities is appealing and
that one option might be that if a contributor was involved in crate sufficiently to review it, then
perhaps they should get rustc r+ too or that out-of-tree crates might offer a way to do more
fine-grained notions of r+. @pnkfelix suggested that long-term we might want in-tree
tests of these crates to guarantee that they maintain compatibility.

@mw then proposed walking through an example - the self-profile working group
will likely create a mildly sophisticated on-disk string table that will be shared between rustc
and external tooling created by the working group and that this would be an ideal candidate for an
out-of-tree crate.

This raised the many questions. Do compiler team members have the rights to create repositories in
the rust-lang organization? Is it best to start in the rust-lang organization or prototype in a
user-hosted repository? What do we name these crates? Is there guidance on adding crates.io
dependencies to rustc?

@davidtwco suggested that we would need to collate a list of things that would need to
be added to a repository (licenses, code of conduct, etc.) and things that would need set up (bors,
highfive, etc).

@wesleywiser mentioned that while anyone on the compiler team should be able to r+,
only those actively involved should be on the highfive rotation. @nikomatsakis
noted that it would be great for active working group members to have the opportunity to gain r+.

@mw asked if we would set up community infrastructure (such as Zulip instances or streams) for
the out-of-tree crates, but @nikomatsakis thought it best that this be tied to
the community infrastructure already in-place for the working group.

Then, @nikomatsakis re-raised the question of whether these out-of-tree crates are
best kept in the rust-lang organization. @mw suggested that this is best as it simplifies
bors and highfive integration and would use existing team permissions. @pnkfelix asked
how challenging migration from a personal repository to the rust-lang organization is. On this
point, @nikomatsakis said that transferring is easy but messy - transferring can
leave a redirect behind which will conflict with the default name of a fork, which requires an email
to GitHub to resolve.

@davidtwco interjected that it should be made clear what team and working group is
responsible for any repository we create in the rust-lang organization.
@nikomatsakis suggested a template for the README of out-of-tree crates would be
valuable and should point to the responsible working group.

Changing gears, @mw asked if there were any example workflows for handling changes in external
crates to which @nikomatsakis suggested that a rustc-dev-guide chapter on that and
how to make local copies for debugging would be necessary. @davidtwco asked what
information should be kept in the policy document and what information should be kept in the
rustc-dev-guide. There was some consensus that setting up a new out-of-tree crate is policy and working
with one is for the guide.

@mw then raised the question of where consent of compiler team members should be gathered
before creating new out-of-tree crates. @pnkfelix suggested it could be a triage meeting
item and @nikomatsakis suggested using @rfcbot and that the general presumption
would be that if a compiler team member wanted to create a out-of-tree crate then it would probably
be a good idea. @mw mused that something asynchronous would be preferred and that agreement
within the working group itself should come first. @nikomatsakis suggested that
the goals of approval should be discussed, such as helping to pick a suitable name.

@mw suggested that a list of these crates would be valuable and there was agreement that it
should not be in rust-lang/rust to avoid the bors queue. There was some discussion about searching
for issues across repositories in an organization.

@davidtwco then agreed to write these minutes and his hands are starting to get tired
at this point.

File: content/minutes/steering-meeting/2019-03-22.md

title: 2019.03.22 type: docs

2019.03.22

Zulip topic.

Announcements

oli-obk
mentions that wg-diagnostics is planning to use the annotate-snippets crate within rustc.
nikomatsakis
is planning, as part of the @WG-meta working group, an RFC that will propose more intermediate levels for the compiler team to recognize committed, regular contributors.
Zoxc
almost has a series of PRs pushing queries all the way back to parsing.
davidtwco
has, after numerous attempts, managed to produce a have a working branch
that unblocks RFC 2008.
pnkfelix
applauds oli-obk's fixmeh project, which lists all the FIXME comments in the compiler.

Main topic

Our main topic of the day was enumerating the needs of the compiler team:

Can we try to enumerate what are some of things the compiler team needs to be tracking? What communication is (or should) be happening? I'd like us to be able to come up with a good list of all the things we think should be happening, so that we can compare against our existing processes, and try to ensure they are covered (or adjust the processes as needed).

We produced a paper document that contains a pretty comprehensive listing, sorted into categories.

After other topics that came up along the way:

Ideas for how to have higher-level design meetings
Thoughts about the "office hours" concept
What we can do about status updates
lolbench, the project to measure and track the performance of generated code

File: content/minutes/steering-meeting/2019-04-12.md

title: 2019.04.12 type: docs

2019.04.12

Zulip topic.

Announcements

nikomatsakis
has opened a PR with a draft
RFC on the
compiler-team repository. This RFC describes adding a new level
("compiler team contributors") to recognize people who have been
contributing regularly. It also tries to document a few things about
members etc. Please give it a read!
eddyb
is nearing completion on their work on the symbol naming revamp. It
might even offer some performance
improvements.
nikomatsakis
mentioned that in the wg-traits meeting on
Monday
we plan to discuss how the Chalk crates are setup and the work to
refactor them to make them more friendly for the RLS etc.

Main topic

Our main
topic
of the day was a proposal to add a regular design meeting:

So this was our plan for the main topic today:

After that we'll talk about my proposal for a regular design meeting and perhaps also a bit about working groups and how we feel they are working, what's could be improved, etc.

The goal of the proposal was to have a central place where the team
can review designs and have design discussions. These discussions could
have a few forms:

In some cases, people might bring a thorny problem, looking for help in
finding a design. (Example)
In other cases, people might be bringing an existing idea for
evaluation and broader discussion, looking to reach a
conclusion. (Example)

In each case, there will be a proposal describing the question at hand
and the goals of the meeting. The expectations for this proposal
would vary depending on the kind of
discussion.
For "early stage" discussions, the proposal might be fairly sparse.
(Perhaps like this
example.)
For meetings that aim to reach firm decisions, the proposals would be
expected to contain more
details;
it would be useful to list things the author expects are hard to
do
and also points where there are design decisions to be
made.

The expectation is that these meetings will help us propagate
knowledge of what is going on through the project; they will help
people get feedback on designs (right now, it can be hard to get
people to make time to really talk out a design). Finally, they can
help newcomers to get more familiar with
things,
since people can lurk in the meetings and soak up details (not to
mention the paper
trail).

While the basic idea of the proposal was popular, we wound up changing
some of the details for how it would run. We settled ultimately on
reusing the slot for the existing steering meeting (Friday
mornings, Boston time) but making it weekly.

Every Nth meeting (likely 3 or 6) would be a designated steering
meeting. The role of these steering meetings it so select the next
N-1 topics that will be discussed (from a list of proposals). In
between meetings, people can add meeting proposals to the list for the
next round.

When we make these decisions, we would be trying to take a "global
view".
In particular, we don't want to be doing designs unless there are
people who will put them into
practice. This
probably means that we will prioritize design questions arising from
existing working groups, but we might also take up design questions
that might become the basis for a new working group, as long as there
are people who would want to take part.

We also spent some time discussing the idea of a mentoring or
'internship'
program
for helping "frequent contributors" make the jump to "compiler team
member". The designs that we discuss in this meeting would hopefully
wind up detailed enough that frequent contributors can pick them up
and implement
them,
even if the people making the design are busy with other things.
Similarly, design documents can become the basis for rustc-dev-guide
chapters.

File: content/minutes/steering-meeting/_index.md

title: Steering Meeting type: docs

Steering Meetings

File: content/minutes/triage-meeting/2019-04-11.md

title: 2019.04.11 type: docs

2019-04-11

Volunteers wanted

seeking help on resolving "[firefox] error: relocation refers to local symbol "" [12], which is defined in a discarded section"
seeking help on resolving "Compiler panic with generic-typed nested closures"

Backport decisions

No backport decisions were made at this meeting because there were no PRs nominated for backport.

Working group sync

[wg-mir-opt](../../../working-groups/mir-opt/)

The current goal is to make the MIR Place data structure non-recursive (Issue #52708).
This should use less memory, be easier to work with, and probably faster as well.
After that is completed, there are a number of other Place related refactorings and other ideas from the All-Hands meeting that can be worked on.
Link to full discussion

[wg-pipelining](../../../working-groups/pipelining/)

The goal is to allow Cargo to invoke rustc sooner letting downstream crates start compiling while upstream crates do LLVM work (Issue #58465)
This working group is just getting started and had its initial meeting on 4/5.
Link to full discussion

File: content/minutes/triage-meeting/2019-04-18.md

title: 2019.04.18 type: docs

2019-04-18

Volunteers wanted

No issues discussed this week.

Backport decisions

There were no beta backport nominations this week.
"Use informational target machine for metadata" (#58605) was nominated for backport to stable.
- Nomination was declined

Working group sync

[wg-llvm](../../../working-groups//llvm)

wg-llvm is working on optimizing overflow intrinsics with add and sub instructions.
A PR (#59546) has also been opened to fix a long standing unsoundness issue.
The PR is currently waiting for someone to run lolbench.rs to determine what impact the PR has on generated code performance.

[wg-async-await](../../../working-groups/async-await/)

wg-async-await is currently working though the list of issues blocking stabilization.
- This includes:
The thorniest issue remaining is determining the drop order for unused async function arguments (#54716)
- While investigating this issue, some subtle inconsistencies regarding existing drop order for regular functions were discovered.
- This issue is also not proving to be easy to resolve.

File: content/minutes/triage-meeting/2019-04-25.md

title: 2019.04.25 type: docs

2019-04-25

Volunteers wanted

Somebody needed to reduce #60136
Somebody needed to write documentation #60059

Backport decisions

"Temporarily accept [i|u][32|size] suffixes on a tuple index and warn" #60186
- Accepted
"Revert "compile crates under test w/ -Zemit-stack-sizes" #59911
- Accepted

Working group sync

[wg-rfc-2229](../../../working-groups/rfc-2229/)

RFC 2229 is a proposal to change how closures capture variables.
Instead of capturing the complete variable when a part of it is used (ie foo.bar), only the "path" will be captured by the closure.
This requires a large amount of refactoring in the compiler.
There is currently a PR open which implements part of this but it regresses compiler error messages in some situtations.
There's been ongoing discussion as to next steps and other refactorings some of which have issues with light mentoring steps available.
Link to full discussion

[wg-self-profile](../../../working-groups/self-profile/)

wg-self-profile has been making progress toward the "minimum viable product" tracking issue #58967:
- A crate called measureme has been created which handles serializing and deserializing profiler events to a compact binary format.
- The crate has been integrated with the existing self-profile infrastructure in rustc.
- There are now summarization and flamegraph tools for processing the profiler data in the repo as well.
The next step is to work on integrating the summarization tool with perf.rlo.
Link to full discussion

File: content/minutes/triage-meeting/2019-05-02.md

title: 2019.05.02 type: docs

2019-05-02

Volunteers wanted

call for assistance: Does someone have an ARM (and maybe also MUSL?) host they could use to try to replicate "Stable rustc always panics on arm/musl"

Backport decisions

There were no beta- nor stable-nominations for backport.

Working group sync

[wg-meta](../../../working-groups/meta)

The compiler contributor RFC is in FCP and should be merged soon.
They are trying to get some sort of "design meeting" up and going.
The plan is to discuss that initial proposal in more depth tomorrow at steering meeting, and then get started after that.
The compiler-team repo has a list of issues tracking things that wg-meta would like to get done.
- call for assistance: Convert the compiler-team repo to use GitHub pages

[wg-rls-2.0](../../../working-groups/rls-2.0/)

Lots of progress happening!
Work is being done to expand macros by example so the analyser can understand the generated code.
Chalk is being integrated (#1216).
The rustc lexer code is being extracted into a separate crate (#60261).

File: content/minutes/triage-meeting/2019-05-09.md

title: 2019.05.09 type: docs

2019-05-09

Volunteers wanted

compiler panic "randomly" with incremental build (#60629)
- Help is wanted to reproduce the issue
Regression: Typemap type mismatch in 1.34.0+ (#60375)
- Help is wanted to narrow down the problem

Backport decisions

save-analysis: Fix ICE when processing associated constant (#60649)
- Backport accepted

Working group sync

[wg-nll](../../../working-groups/nll/)

NLL migration mode is enabled on Rust 2015.
The next goal is to remove the old AST-based borrow checker.
Link to full discussion

[wg-diagnostics](../../../working-groups/diagnostics/)

wg-diagnostic's current goal is to pull some of the diagnostic infrastructure out of the compiler and into a reusuable crate.
They are working on organizing their first meeting soon.
Link to full discussion

File: content/minutes/triage-meeting/2019-05-16.md

title: 2019.05.16 type: docs

2019-05-16

Volunteers wanted

request for investigation: "Compiler panic with generic-typed nested closures" (#59494)
request for investigation: "Exponential compile-time and type_length_limit blowup when nesting closure wrappers" (#54540)
- eddyb has excellent notes here on the fundamental problem
request for investigation: "llvm lint: "Undefined behavior: Call argument type mismatches callee parameter type" with mixing debug and release" (#48310)
- specifically, we need to resolve whether this is in fact an LLVM bug
request for investigation: "ICE with unsized associated type" (#60431)

Backport decisions

save-analysis: Pull associated type definition using qpath_def" (#59894)
- Backport accepted
Use delay_span_bug for error cases when checking AnonConst parent (#60710)
- Backport accepted
Use delay_span_bug for "Failed to unify obligation" (#60644)
- Backport accepted
Instead of ICEing on incorrect pattern, use delay_span_bug (#60641)
- Backport accepted
conditionally modify darwin targets to macosx targets with versions (#60378)
- Backport declined
default to $ARCH-apple-macosx10.7.0 LLVM triple for darwin targets (#60788)
- Backport declined

Working group sync

We ran out of time to hear from working groups in this meeting.

File: content/minutes/triage-meeting/2019-05-23.md

title: 2019.05.23 type: docs

2019-05-23

Volunteers wanted

No issues discussed this week.

Backport decisions

"Fix ICE with inconsistent macro matchers" (#61046)
- Backport accepted
"debuginfo: Revert to old/more verbose behavior for -Cdebuginfo=1" (#61007)
- Backport accepted

Working group sync

[wg-pgo](../../../working-groups/pgo/)

PGO stands for Profiler Guided Optimization.
Code is instrumented and then run. The profilng data gathered at runtime is then used in subsequent compilations to guide optimizations.
PGO support is almost complete!
- The remaining work is mostly just around the UI side of things: documenting compiler flags, moving flags from -Z to -C.

File: content/minutes/triage-meeting/2019-06-06.md

title: 2019.06.06 type: docs

2019-06-06

Volunteers wanted

No issues discussed this week.

Backport decisions

Fix regression 61475 #61500
- Backport approved
Turn turbo :fish: :ice_cream: into an error #61189
- Backport approved
Fix overflowing literal lint in loops #61098
- Backport approved
Fix more escaping ReScopes #60765
- Backport approved
Constrain all regions in the concrete type for an opaque type #60449
- Backport approved

Working group sync

[wg-mir-opt](../../../working-groups/mir-opt/)

const propagation (by Wesley Wiser) on mir opt level 2
- some great compile-time speedups due to llvm and later MIR passes having less to do
- Perf (active PR) has some regressions (up to 4% though)
rewrite loads of recursive Place algorithms imperatively (by Santiago Pastorino)
WIP PR by Santiago Pastorino for making Place a struct instead of an enum (preliminary work for making projections a slice instead of a recursive enum)

[wg-pgo](../../../working-groups/pgo/)

Nothing new to report

File: content/minutes/triage-meeting/2019-06-13.md

title: 2019.06.13 type: docs

2019-06-13

Volunteers wanted

No issues discussed this week.

Backport decisions

Handle index out of bound errors during const eval without panic #61598
- Approved for beta backport.

Working group sync

[wg-llvm](../../../working-groups/llvm/)

Nikita Popov fixed a codegen issue with LLVM float min/max intrinsics on X86.
This allowed varkor to switch our min/max functions to use those intrinsics.
Nikita Popov is working on getting CVP nowrap inference reenabled in LLVM.

[wg-async-foundations](../../../working-groups/async-await/)

The list of blocking issues is shrinking!
Two major issues have had signficant work done on them:
- nikomatsakis posted a PR which closes a number of lifetime related issues.
- tmandry is working on shrinking the size of futures.
Work is commencing on the "async book" which will help explain how async I/O works and the async-await feature in particular.
Many more tests need to be written around the feature.
- This might be a good way to get involved with the working group!

File: content/minutes/triage-meeting/2019-06-20.md

title: 2019.06.20 type: docs

2019-06-20

Volunteers wanted

No issues discussed this week.

Backport decisions

Beta: "resolve: Temporarily make panic available with no_implicit_prelude" #61971
- Backport accepted
Beta: "Fix ICE involving mut references" #61947
- Backport accepted
Beta: "rustc_typeck: correctly compute Substs for Res::SelfCtor." #61896
- Backport accepted
Beta: "Fix rust-lldb wrapper scripts." #61827
- Backport declined
Beta: "Fix cfg(test) build for x86_64-fortanix-unknown-sgx" #61503
- Backport accepted
Beta: "Dont ICE on an attempt to use GAT without feature gate" #61118
- Backport accepted
Beta: "Limit dylib symbols" #59752
- Backport accepted
Beta: "create a "provisional cache" to restore performance in the case of cycles" #61754
- We're going to wait a week and then revisit this
Beta: "Hygienize macros in the standard library" #61629
- Backport accepted

Working group sync

There wasn't time to check-in with the working groups this week.

File: content/minutes/triage-meeting/2019-06-27.md

title: 2019.06.27 type: docs

2019-06-27

The meeting largely consisted of discussion of a few nominated issues:

Rustdoc recursion limit issue
- Assigned to @nagisa
Incorrect span / broken rustfix: help: use dyn: dyn #[dom_struct]
- @davidtwco is assigned and working on a fix.
Soundness hole in pattern matching on enums with an uninhabited variant
- Assigned to @eddyb.
Self as default type isnt typechecked
- Under discussion at a future T-Lang meeting.
Incremental compilation results in linker error when method use is removed
- Assigned to @Zoxc tentatively as @mw is on leave.
Coherence can be bypassed by an indirect impl for a trait object
- Assigned to @centril and it has a T-Lang meeting slot.
The compiler should report publicly exported type names if possible
- There was some discussion about why this is difficult currently and possible approaches to resolving this issue.
- No firm conclusion was reached so the nomination label was left on the issue and it can be picked back up next week.
"immutable field" errors are confusing when the handle is mutable, should describe why it is immutable
- This issue is P-high for WG-diagnostics but not necessarily the entire compiler team.

File: content/minutes/triage-meeting/2019-07-04.md

title: 2019.07.04 type: docs

2019-07-04

Volunteers wanted

No issues discussed this week.

Backport decisions

There were no backport nominations this week.

Working group sync

wg-meta

Conferences, Mozilla All-Hands, and other activities have been keeping members of the group busy recently.
So there's been a slight stall in activity.
The compiler team website is nearly completed thanks to @Federico Carrone and @catalinasy!

File: content/minutes/triage-meeting/2019-07-11.md

title: 2019.07.11 type: docs

2019-07-11

Volunteers wanted

No issues discussed this week.

Backport nominations

Raise the default recursion limit to 128
- Approved for beta backport
Fix ICEs when Self is used in type aliases
- Approved for beta backport
Emit warning when trying to use PGO in conjunction with unwinding on Windows
- On hold while @wesleywiser looks into alternative approaches

Working group sync

wg-pipelining

Tracking issue #60988
@alexcrichton is proposing we stabilize the feature and turn it on for everyone by default.
There are two main parts to this:
- A --json flag which will replace --json-rendered
- JSON notifications when artifacts are complete

File: content/minutes/triage-meeting/2019-07-18.md

title: 2019.07.18 type: docs

2019-07-18

Volunteers wanted

No issues discussed this week.

Backport nominations

Fix #62660
- Approved for beta backport
- Declined for stable backport
Cancel unemitted diagnostics during error recovery
- Approved for beta backport
Handle errors during error recovery gracefully
- Approved for beta backport
Correctly break out of recovery loop
- Approved for beta backport
- Declined for stable backport
Emit warning when trying to use PGO in conjunction with unwinding on Windows
- Declined because it's supersceeded by #62615
Only error about MSVC + PGO + unwind if we're generating code
- Approved for beta backport
rustc_target: avoid negative register counts in the SysV x86_64 ABI
- Approved for beta backport

Working group sync

There wasn't time for a working group sync this week.

File: content/minutes/triage-meeting/2019-07-25.md

title: 2019.07.18 type: docs

2019-07-25

Volunteers wanted

#60297 - we are looking for someone with access to ARM/musl to help debug this
#62896 and #62932 - there are some embedded failures, quite possibly related to LLVM, that could use bisection

Backport nominations

Make the parser TokenStream more resilient after mismatched delimiter recovery
- Approved for beta backport

Working group sync

No working group sync this week.

File: content/minutes/triage-meeting/2019-08-01.md

title: 2019.08.01 type: docs

2019-08-01

Volunteers wanted

No issues discussed this week.

Backport decisions

There were no pending backport decisions this week.

Working group sync

No working group sync this week.

File: content/minutes/triage-meeting/2019-08-15.md

title: 2019.08.15 type: docs

2019-08-15

Volunteers wanted

#60297 - we are looking for someone with access to ARM/musl to help debug this

Backport nominations

#63508 Do not ICE when synthesizing spans falling inside unicode chars
- Beta backport accepted

Working group sync

[wg-rustc-dev-guide](https://rust-lang.github.io/compiler-team/working-groups/rustc-dev-guide/)

wg-rustc-dev-guide's objective is to make the compiler easier to learn by ensuring that rustc-dev-guide is "complete".

Currently they're working on:

Getting video lectures about parts of the compiler from people who know them
Turning existing video lectures into content for the rustc-dev-guide
Triaging and working on issues and pull requests for rustc-dev-guide content

Help is always wanted!

File: content/minutes/triage-meeting/2019-08-29.md

title: 2019.08.29 type: docs

2019-08-29

Volunteers wanted

We've had a few issues recently related to upgrading LLVM.
We'd like to create a new working group which will focus on fixing ICEs and miscompilations caused by LLVM.

Backport nominations

Account for doc comments coming from proc macros without spans #63930
- Accepted for beta backport
fix nounwind attribute logic #63909
- Declined for beta backport
Fix nested eager expansions in arguments of format_args #63717
- Accepted for beta backport
Don't access a static just for its size and alignment #62982
- Accepted for beta backport
debuginfo: give unique names to closure and generator types #63875
- Accepted for beta backport

Working group sync

[wg-mir-opt](https://rust-lang.github.io/compiler-team/working-groups/mir-opt/)

@spastorino has gotten Place 2.0 (a reimagining of mir::Place using slices for projections) with only a slight regression in memory usage on one compiler benchmark.
- They're currently working on fixing that regression.
@wesleywiser split promoted items out of mir::Body to prevent cycles in the optimization pipeline.
- They're continuing to work on making the const_prop pass more powerful.

[wg-nll](https://rust-lang.github.io/compiler-team/working-groups/nll/)

A few soundess "fixed-by-nll" issues were closed recently.
Work is ongoing to stabilize bind-by-move pattern guards.

[wg-parallel-rustc](https://rust-lang.github.io/compiler-team/working-groups/parallel-rustc/)

@aturon is doing an audit of related PRs (accepted and pending) with a focus on shared state and lock semantics with multithreaded rustc.
@aturon is also going to be working on reducing locking and shared mutable state that exists outside the query system.

File: content/minutes/triage-meeting/2019-09-05.md

title: 2019.09.05 type: docs

2019-09-05

Volunteers wanted

Backport nominations

Update xLTO compatibility table in rustc book #64092
- Approved for beta backport
Support "soft" feature-gating using a lint #64066
- Approved for beta backport
pprust: Do not print spaces before some tokens #63897
- Approved for beta backport

Working group sync

No working group sync this week.

File: content/minutes/triage-meeting/2019-09-12.md

title: 2019.09.12 type: docs

2019-09-12

Volunteers wanted

We are looking for someone with access to an arm/musl system to help troubleshoot #60297

Backport nominations

Update LLVM submodule #64317
- Approved for beta backport
fix nounwind attribute logic #63909
- No conclusion reached yet. We'll revisit next week.
pprust: Do not print spaces before some tokens #63897
- Approved for beta backport

Working group sync

[wg-polonius](https://rust-lang.github.io/compiler-team/working-groups/polonius/)

After a long hiatus, this working group is being rebooted with a focus on expanding Polonius coverage to include initialization checking and region subtyping in addition to borrow checking.
Work is also proceceeding on a book which will document the rules for Polonius. The hope is this will both make it easier to understand Polonius and also help make it easier for new contributors to get involved.
@Albin Stjerna is also working on a master's thesis which overlaps with the Polonius work.
@Aaron Weiss and @nikomatsakis realized that there maybe opportunities to make Polonius more precise. Related video chat
@lqd has been working on a rustc compare mode which will check Polonius against the current borrow checker.

@Matthew Jasper and @lqd have made a number of fixes and cleanups so nearly all of the rustc tests pass under the Polonius compare-mode.

Some general performance work has been done but there's more to do especially around large constants.
Some vocabularly is also being decided on.

[wg-rfc-2229](https://rust-lang.github.io/compiler-team/working-groups/rfc-2229/)

RFC 2229 changes closures so they don't capture entire variables, but rather subpaths.
@blitzerr has been working on a refactoring related to how closure substitutions work. This is mostly done but it's hit some issues related to diagnostics and needs also needs to be rebased.
Work is ongoing.

File: content/minutes/triage-meeting/2019-09-19.md

title: 2019.09.19 type: docs

2019-09-19

Volunteers wanted

"Better error message when attempting to instantiate tuple structs with private fields" #58017
- Volunteer wanted to bisect this issue.

Backport nominations

"fn ptr is structural match" #64431
- Accepted for beta backport
"fix nounwind attribute logic" #63909
- Declined for beta backport

Working group sync

[wg-self-profile](https://rust-lang.github.io/compiler-team/working-groups/self-profile/)

Three main things being worked on:

Getting the "minimum viable product" done as planned #58967
Improving the ergonomics of self-profiling

we want to collect use cases on irlo and then try to come up with good, tooling assisted workflows for those use cases

Use self-profiling to implement a feature that allows to attribute compile time to specific parts of the source code

this feature is currently still in the brainstorming phase
comments welcome https://github.com/rust-lang/measureme/issues/51

[wg-rls-2.0](https://rust-lang.github.io/compiler-team/working-groups/rls-2.0/)

Chalk has been integrated, a number of performance problems have been fixed, and now it works great!
Macro by example and associated macro resolution work well enough to cover significant subset of the real world code
We added a simple hack to show compiler errors by just running cargo check
First win of librarification: rustc and rust-analyzer share the same lexer code!
After slowing down during August, they are starting the next sprint. The goal is to library-ify type-relating things: so, chalk + unification + a definition of Ty

File: content/minutes/triage-meeting/2019-09-26.md

title: 2019.09.26 type: docs

2019-09-26

Volunteers wanted

Help wanted: proc-macro param attrs dropping first attrs in impl fns #64682
Help wanted: error: internal compiler error: unexpected panic: inconsistent resolution for a macro #64803

Backport nominations

Fix #64744. Account for the Zero sub-pattern case. #64748
- Approved for beta backport
Rustdoc render async function re-export #64599
- Approved for beta backport

Working group sync

[wg-traits](https://rust-lang.github.io/compiler-team/working-groups/traits/)

wg-traits is spinning back up and has a number of projects currently being worked on:

More Chalk work
Investigating lazy normalization
Enabling trait upcasts
Resolving dyn Trait coherence and well-formedness issues
Work is progressing to allow stabilization of the ! type

File: content/minutes/triage-meeting/2019-10-10.md

title: 2019.10.10 type: docs

2019-10-10

Announcements

@centril is splitting libsyntax into data and logic parts which will decrease the amount of code librustc depends on.
@nagisa a cross-platform version of stacker which will allow us to avoid stack overflows in rustc.
There is a compiler team design meeting scheduled for tomorrow (2019-10-11) to discuss some of @Zoxc's PRs.
@pnfelix is revising the code that handles structural match checking.
@nikomatsakis has a PR (#65232) up which helps us get closer to lazy normalization.
wg-traits is going to start holding a weekly "office hours" video call to help answer questions and teach people about trait system internals.

Volunteers wanted

Backport nominations

Ensure that associated async fns have unique fresh param names #65142
- Approved for beta backport
Account for macro invocation in let mut $pat diagnostic. #65123
- Approved for beta backport
Fix the start/end byte positions in the compiler JSON output #65074
- Declined for beta backport
extract expected return type for async fn generators #64999
- Declined for beta backport

Working group sync

[wg-rustc-dev-guide](https://rust-lang.github.io/compiler-team/working-groups/rustc-dev-guide/)

wg-rustc-dev-guide has been working on transcribing videos from the compiler lecture series into rustc-dev-guide chapters.
Originally, individuals were assigned one or lectures to complete but that hasn't worked very well.
Recently, they're trying to work on one video at a time as a team with much better results.
There's a PR open for a new chapter based on the ty lecture.

[wg-llvm](https://rust-lang.github.io/compiler-team/working-groups/llvm/)

rustc has upgraded to the LLVM 9 release;
- Which allows us to replace some the emscripten stuff with LLVM’s toolchain. The less emscripten we have to deal with, the better!
- As part of these upgrades we will likely end up dropping support for the super old LLVM 6, which in turn allows us to stop building the unnecessary Go & OCaml bindings to LLVM, which in turn helps rustc build times slightly.
People are also working on enabling use of the new pass manager, which might give us some tangible code quality improvements over the status quo.

File: content/minutes/triage-meeting/2019-10-17.md

title: 2019.10.17 type: docs

2017-10-17

Announcements

Design meeting on debuginfo strategy tomorrow (2019-10-18) on Zulip.
The traits working group had a design meeting yesterday on lazy normalization.
@nikomatsakis has been looking into lazy normalization specifically for constants and hopes to have some notes to share soon.
The LLVM ICE-breakers working group is nearly ready to go live.
@spastorino has nearly finished interning Place projections. PR 1, PR2
@centril is continuing to work on splitting libsyntax apart

Volunteers wanted

Segfault compiling libc on armv7-unknown-linux-gnueabihf #62896
Couldn't load codegen backend on windows-gnu #61561

Backport nominations

Add troubleshooting section to PGO chapter in rustc book #65402
- Approved for beta backport
Optimize try_expand_impl_trait_type #65293
- Approved for beta backport
Use precalculated dominators in explain_borrow #65172
- Approved for beta backport

Working group sync

[wg-mir-opt](https://rust-lang.github.io/compiler-team/working-groups/mir-opt/)

@wesleywiser Moved promoted MIR out of mir::Body #63580
@wesleywiser Reimplemented the ConstProp optimization pass on top of the existing const eval code #64419
@spastorino Converted mir::Place to no longer be a recursive data structure #63420
There is ongoing work in both const prop and place refactorings.
We've figured out a plan for "weird" place projections like dereferences (virtual locals).

[wg-meta](https://rust-lang.github.io/compiler-team/working-groups/meta/)

The Inside Rust blog has launched.
The ICE-Breaker group has been formed!
The LLVM ICE-Breaker group is also being formed.

File: content/minutes/triage-meeting/2019-10-24.md

title: 2019.10.24 type: docs

2019-10-24

Announcments

@simulacrum landed the rustc-dev split PR which decreases the size of the rustc-dev rustup component #65474

Backport nominations

Avoid ICE when adjusting bad self ty #65755
- Accepted for beta and stable backport
Avoid ICE when checking Destination of break inside a closure #65518
- Accepted for beta and stable backport
save-analysis: Nest tables when processing impl block definitions #65511
- Accepted for beta backport
save-analysis: Don't ICE when resolving qualified type paths in struct members #65353
- Accepted for beta backport

Working group sync

[wg-nll](https://rust-lang.github.io/compiler-team/working-groups/nll/)

Rust 1.40 (current nightly) will be the first stable release without the HIR borrow checker.
This means Non Lexical Lifetimes will be available on Rust 2015.
At this point, wg-nll has completed its purpose and will be disbanded.
- wg-polonius is still going strong though!

[wg-parallel-rustc](https://rust-lang.github.io/compiler-team/working-groups/parallel-rustc/)

Work is proceeding slowly but steadily.
There are regular triage meetings every Monday on the compiler calendar.
The current goal is to refactor/audit compiler locks and other parts of the parallel query system and to investigate improving performance.
The current MVP is to do high-level parallelization in a few critical places such as typechecking & linting.
Currently, we're not seeing the speedups we want from parallelization but work is ongoing to remove locking and improve the jobserver integration.
Video recording of last meeting

File: content/minutes/triage-meeting/2019-10-31.md

title: 2019.10.31 type: docs

2019-10-31

Announcments

Rust 1.39 ships on Thursday!

Backport nominations

"save-analysis: Account for async desugaring in async fn return types" #65936
- Accepted for beta backport
"resolve: Turn the "non-empty glob must import something" error into a lint" #65539
- Accepted for beta backport

Working group sync

[wg-pgo](https://rust-lang.github.io/compiler-team/working-groups/pgo/)

PGO is available in the stable compiler. Docs are in the rustc-dev-guide and the rustc-book
Unfortunately we don't observe significant performance gains from applying it (except for tiny synthetic test cases).
@michaelwoerister asked for people to try PGO on irlo but haven't gotten a lot of feedback.
There is one interesting theory that Rust doesn't profit as much as C++ because it defaults to having fewer compilation units and thus can make better inlining decisions in the non-PGO case.
The working group will probably wind down as @michaelwoerister don't have time to pursue further and there are no actual known issues in the implementation.

[wg-pipelining](https://rust-lang.github.io/compiler-team/working-groups/pipelining/)

Pipelining support has shipped in Rust 1.38.
The working group is winding down.