Here's my #rust2018 post to add to the batch.

The past few years, I've been lucky to be part of the Rust community survey process. In 2016, this data helped show the need to improve usability. We had good foundations, but they were too hard to get at. In 2017, we saw a renewed need to reach out to companies, to improve documentation, and a continued importance of pushing to making Rust more usable.

These surveys were a key part of seeing big issues, but it got me thinking. Are there better ways to get and use data? Can we see Rust more clearly -- for what we do well and where we need to improve -- in the future?

When you do performance tuning, you benchmark and run perf profilers to get at the root of the problem. What if we did the equivalent for Rust as a whole?

We already measure through the surveys. The problem is, we can only really measure once or twice a year. This results in waiting long periods of time to see if we successfully addressed an issue. The delay may mean we don't catch new issues in time or we may fix a problem only to find out we didn't address the underlying cause.

Getting better data, sooner

What if we had a way to have better data sooner? There are a few ways we could do this.

Smaller surveys

The first thing we could do is put out smaller surveys and use them as a quick check-in. They might target specific features in a "did this change solve your issue?" way. We do a bit of this in the RFC process already. People who know about a particular RFC can jump in and comment on whether or not it the approach works for them.

While we want to continue using RFCs for this feedback, we could extend what we do here. For example, a blog post could talk about a group of RFCs working together to address one core problem. We could have a quick survey as part of the post to gather feedback on whether the combination of changes does what we hoped.

This is still pretty manual, though, and is focused developers actively reading RFCs and blogs and volunteering their time, which may be limited. Are there ways we can get more people involved across a wider range of changes?

Usability studies

There's another avenue we could explore: usability studies. Work on software development in the past has shown there's naturally a distance between what people think they spend the most time on and what they actually do spend the most time on. Self-reporting often misses these insights.

When you actually watch someone working, issues with usability become more obvious.

With the rise of coding streams, we already have some usability data already. Perhaps there is a way to leverage the same tools so that we can perform studies remotely. The end result of these could more directly answer how workflows improve, and give us a clearer insight into both the Rust learning process and how more experienced Rust users use Rust in their work.

Opt-in statistics

What if Rust developers could opt-in to some statistics gathered from the tools themselves? This would let Rust developers help the Rust project as part of their everyday work.

So it's clear, let me state I'm a strong privacy advocate. I have no desire to read people's private source code, to try to "guess" what they're doing, or anything remotely like that. Instead, here are some ideas we could be looking at that are helpful while not being invasive:

• What error messages are people hitting? How often?
• Roughly how much time is spent working in Rust?
• How long are build times compared to the lines of code? How much is incremental compilation helping?
• Roughly how much are particular language features being used?

I believe there's a way to we can gather data like this in a way that is both opt-in and preserves the privacy of the developer.

Transparency

An important aspect of "better data, sooner" is transparency.

If we're careful, we can and should make the data we gather public. It gives contributors a window into the Rust world in more depth than we've previously had.

Imagine a live "board" where you could go and see Rust being used in real time, seeing what errors are occuring the most or if the latest fixes to nightly improved build times in the wild. This data could help not only people putting code in the compiler, but also folks writing RFCs, documentation, and more.

Measure twice, cut once

When I was growing up, my dad would do various woodworking projects around the house. When I'd help, he would remind me to "measure twice, cut once". You didn't want to waste the effort it took to cut the wood if you couldn't use the result.

We can take a similar attitude for Rust in 2018. There are a lot of things we could do. Lots of initiatives. Lots of projects. Can we come up with ways to measure twice before we invest the time? I think we can, and Rust will be better for it.

Here's to a happy, successful Rust in 2018!