🏃 In-progress and incomplete: still trying to wrap my head around much of it, coordinating with partner teams to enable the right interfaces, etc., and there are a bunch of unanswered Qs, but just wanted to get the conversation started.
###Goal: Build Best-of-Breed Node.js Developer Experiences In particular, we'll be focusing on inner-loop development, but at the same time we'll want to ensure that the same awesome experience transcends product/service boundaries.
####Table Stakes First and foremost, it's important that we agree on a set of table stakes. These are the experiences that are really foundational - without them, it's a no-op.
- Debugging (including sourcemaps) needs to be consistent and reliable
- Really easy and lightweight project configuration - minimize time-to-F5
- Consistent and just-works IntelliSense with a gracefully degrading experience.
- Keep up with the constantly changing landscape: Node, JS, TS, Babel, JSX, etc.
####Execution One of the challenges about building tooling for Node.js is that we're building on top of a fast-moving platform, so it becomes that much more important to ensure that our efforts are aligned engineering-wise. In that vein, we're currently working to see how we can best leverage and consolidate our Node.js tooling efforts across VS & VSCode to enable us to move faster on both products and also provide a more seamless round-tripping experience between the two.
The basic idea is that we have a few stages to roll out here:
- Node+VSCode and Node+VS are each at different stages in their development and have their own set of immediate priorities to execute on.
- Combined engineering efforts, basically how do we share as much code as possible? This'll help us move faster as a team, and also help us enable a more seamless experience across product boundaries.
- new features and future scenarios, which will be built from the beginning with the intent of sharing as much code as possible.
The next several sections dig deeper into each one of these areas. Also note that the intent of this document is to focus on some of the 'big rocks' rather than iterative changes, which, of course, we will continue to execute on in parallel.
Mostly prepping for GA, componentizing & producing solid platform, patching issues, or low-hanging fruit features. VSCode's roadmap is posted here: https://github.com/Microsoft/vscode/wiki/Roadmap
We recently release v1.1, so we are starting to investigate some of the big-ticket items on our plate. Added the beginnings of a roadmap below, but this is pretty raw at the moment, and still fleshing out some details before posting more broadly.
- Ship NTVS 1.1.1
- Adopt file/folder-based project system
- works better with node.js workflow
- Tightly integrated task runner support
- Lightweight installation option
- Migrate to Salsa when we have automatic d.ts acquisition
- helps solve perf issues with analysis engine
- ES6 support
- reduces duplicated effort
- Better consistency w/ other JS-isms
- Dev15 Support
- Appear in VS installer feed to increase discoverability
- Migrate to TypeScript source-map codebase
- reduce duplication of effort
- new features like tsconfig
- Test infrastructure improvements
- Update Web Tools npm component
- Migrate processes & assets through org transition
Goal: share as much code as possible between VS & VSCode-Node efforts, whilst enabling each product to evolve to meet the needs of their own communities.
The same build script would build for both products, and the CI server would take care of both VS & VSCode builds. This would ensure that it is easier for us to achieve our round-tripping and duplication-reduction goals between VS & VSCode, as well as enable us to keep both products updated with the latest changes.
Additionally...
- Shared debugging & sourcemapping codebase to enable us to more easily keep up with node / javascript
- better diagnostics to help us iterate towards the most reliable debugging experience.
- Work with other teams in VS to prevent duplication of effort (e.g. webkit debug engine)
- Easy round-tripping between VSCode & NTVS
At some point, a central "Node.js Tools" repo was proposed because it may enable us to more easily engage with the Node.js segment of the VS & VSCode communities (note that this does not imply that both products will share the same featureset.) While this would make it much easier for us from a technical standpoint, it may actually have a negative effect on the communities themselves - multiple product roadmaps could be confusing, and we don't want one community to feel drowned out by another.
This is meant to help us reason about the current products and better understand potential ways to share code with each of these features. Over time, we will fill in each one of the sections with more details. Also note that this is a very surface-level understanding of the state-of-the-world - e.g. even though both VSCode & VS have debugging & IntelliSense, there are, in fact, feature differences between the two implementations.
| Feature | Visual Studio | VS Code | Partner teams |
|---|---|---|---|
| Project Templates | ✔️ | ❌ | VS Platform |
| launch.json/folder-based project system | ❌ | ✔️ | VS Platform |
| .njsproj/MSBuild project system | ✔️ | ❌ | Python Tools, VS Platform |
| V8 Debugger | ✔️ | ✔️ | VS Debugging, Cordova webkit debug engine, maybe ASP.NET in the future |
| IntelliSense | ✔️ | ✔️ | TypeScript |
| Node.js-specific snippets | ✔️ | ❌ | |
| Source Mapping | ✔️ | ✔️ | TypeScript |
| Task-runner support | ✔️ | ✔️ | Web Tools |
| npm Integration | ✔️ | ❌ | Web Tools |
| Interactive Window | ✔️ | ❌ | Python Tools, Roslyn, etc. |
| Profiling | ✔️ | ❌ | VS Diagnostics, Glimpse? |
| Unit testing | ✔️ | ❌ | Potentially bring this to all JS projects in VS? |
| Deployment | ✔️ | ❌ | Azure Tools |
| Options | ✔️ | ✔️ |
Reduce code duplication, and thereby respond to the following events once, rather than twice:
- V8's protocol changes.
- Enable other JS engines / Chakra-specific functionality (e.g. TTD)
- Node.js version causes breaking changes that we want to detect and warn the user about.
- Aggregate data and enable deeper diagnostics.
- Iterate quickly towards a stable debugging experience
- Source mapping? (are we using this for anything other than debugging?)
Enable the following new scenarios:
- Attach and roam state between multiple toolsets (e.g. Glimpse)
- Inject code during diagnostics (e.g. capturing state of event queue, handle child processes, etc.)
- [REACH] If adopted by other tools like node-inspector, it could enable us to help improve experiences more broadly for TypeScript, Chakra, etc.
- Increased community contributions to NTVS because the service would be written in Node.js
The idea here would be to effectively turn the VS debug adapter into a shim for the VSCode adapter. And in order to enable VS and VSCode to evolve at different rates, we should consider an extensibility model that would enable us to light up new features before they become integrated into VSCode itself.
We could strive for a service-based approach, where instead of actually spinning up the app in question and having a direct connection between VS & V8, we’d instead subscribe to a Node.js service that is handling all of these connections and providing the relevant information.
First step – currently digging into the VS / VSCode debugging interfaces to get a very thorough understanding of the two architectures and get more concrete on what it might mean to share code between the two. Additionally understanding how the functionality differs between the two will also be important.
We should also ensure to take advantage of the work of other teams who may have started or plan to start working through similar issues (powershell, python, cpp??), so hopefully we can get some ideas from them, but it also sounds like those teams are working with a very different set of constraints. For instance, unlike other teams, we’ll ideally want the bulk of our code to be written in Node.js if possible. Additionally, we have a bit of additional complexity with things like source-mapping. While we’re thinking through this too, I want to see how we could handle things like child processes better during debugging – currently those open up separate debug ports and it can be problematic when we fail to attach to them. I suspect we’ll need to intercept whichever low level functions fork the process in order to understand both when new ports are opened up and also what the port number is.
- Async debugging
- Time-travel debugging
- Glimpse
- Language service features / extensibility model
- Platform / API improvements -> VSCode & VS
At some point, we started hitting flakey debugging issues with NTVS that were:
- Frustrating for users
- Impossible for us to reproduce
- Impossible for the user to consistently reproduce
Given that debugging is one of our key value-adds, it's more critical than ever to ensure we can produce a solid experience. So we added a Live Diagnostics Pane that users could optionally enable, and then just copy the logs when running into these issues. It was easy for users to grab the relevant logs to their session, as they are all clearly demarcated, and it enabled us to resolve a number of threading issues in a very short period of time based off of just diffing the logs against a working version and then reasoning about the code itself.
This feature was really successful in helping us diagnose issues, so it would be great to see just how far we can run with it. Some ideas include:
- making it more discoverable - detecting when debugging is not working, and providing the right interfaces to diagnose and provide us with the right information to iterate on the experience. Maybe something similar to how Windows sometimes detects something wonky with the installation, so it'll ask you if it installed OK.
- add live-logging to collect relevant source-map info?
- There will be a disparity in featureset between VS & VSCode. The question, then, becomes what that is, and whether we ever plan on resolving it. In order to answer this Q, we'll need to consider required engineering effort, UI paradigms, worflows, etc. Either way, if possible, we should aim to design things in a way that is portable.
- The modularity Q: are there things that belong in a separate extension, rather than in the main codebase.
- Optimistically, we can say "hey, we're doing the same thing in two different places... what if we just built one thing that plugged into those two places?" And all this consolidation work sounds great in theory, but the fact is that this won't be easy and so far we haven't landed on, tangibly, what this means.
- Timelines? Currently working on fleshing this out. Due to all the partner team dependencies, it may get complicated...
- To be continued...
In addition to this engineering work, we should continue to focus on recruiting / growing the team, as well as continuing to engage with the community in person, through blogs, videos, the guidelines repo, contributions to node, etc. as well as working with partner teams within Microsoft to ensure we're building best-of-breed experiences that extend beyond just inner-loop dev tooling (e.g. deployment, monitoring, etc.)
Regarding Node debugging in VS Code, we can prioritize the VS Code Debug Issues to select the initial set of improvements.