Features

Jump to definition
- Uses Knowledge about all top level functions/values, type aliases and ADTs.(including 3rd party) Needs file and location info
Show doc and type annotations
- Uses: Same as jump to + raw doc strings (and type info where applicable)
Context sensitive autocompletions
- Uses: Same as jump to + all modules + what they expose + details about params / destructured params and aliases
- Wants: Details about function bodies symbols (let blocks, pattern matches etc etc)
Find symbol
- Uses: Same as jump to, but not accurate due to no function body parsing and type inference
- Wants: Smarter type inference to get close to or reach 100% accurat hit ratio
Doc search
- Uses: Doc strings for all exposed functions in all modules available to a given project
Context sensitive autocompletions
- Uses: Same as jump to, filtering based on knowledge about what context you are in (import module name, exposing clause, function body etc)
- Wants: More :-)
Test support
- Uses: Functions identified as test functions based on function annotaation
Interactive Module diagram
- Usess: All project modules, their import relations and directory
Highlight exposeds
- Uses: Which top level definitions are exposed and which are not
Simple refactor support
- Quick import
  - Uses: Available modules and their exposed top level definitions
- Expose/unexpose
  - Uses: Same as highlight exposeds
Linting
- Uses: elm-make
- Wants: More structured/defined output
Quickfixes
- Uses: elm-make output
- Wants: More structured/defined output
Package doc preview
- Uses:Elm make --docs
Package manager
- Uses: all-docs endpoint, elm-package.json, exact-dependencies.json
Package dependency diagram
- Uses: Same as package manager + elm-package.json files for project and 3rd party modules
elm-reactor
- Uses: Custom port range allocation, elm-reactor binary
elm-format
- Uses: stdin/stdout for elm-format
elm-repl
- Uses : stdin/stdout for elm-repl process
select expression
- Uses: Custom determination of top level expression bodies
- Wants: IntelliJ style expand selection

Feature details

Jump to definition
- Jump to any top-level defintion from a given symbol - Need to traverse all imported top-level definitions (including aliasing or by module name prefixing) and module local top level definitions to find a match. If one found we have the ast node for that which has the location information, so that's enough info to open that file at the correct line level. So this works for both project sources and 3rd party package sources
- Since the parser also fully parses ADTs you can also jump to a specific constructor
- Since the parser also fully parses type aliases, you can jump to a specific field definition from symbols within a function body, given that function has a type annotation.
- Jump to from function body symbols also works for pattern matched parameters so that you can jump to a say a record field that is a parameter to a type constructor. This is not solved recursivly so it only supports the most typical cases I have come accross/use my self !
Show doc and type annotations
- Very similar to jump to defintion in terms of lookup. I can display doc comments (rendered markdown in the editor since the editor supports html display) for top level definitions that supports doc comments. For record fields and type constructors it will display the given defintion/types that is defined as (where applicable).
Find symbol - Locate all instances/usages of a given symbol within your project. This is a step up from textual search, but since it's not parsing function bodies it's not bullet proof and doesn't cover shadowing or record update syntax etc. Works ok, but not well enough to be used for refactor-rename imho.
Doc search - Search for functions/symbols in your project and accross 3rd party packages your project depend on. Again relies on parsing all relevant files and their doc strings.
Context sensitive autocompletions
- Module names in import clause - When at "import <X>" it suggests only valid Module names you may import, and it also filters out modules you have already imported. For this we need all exposed Modules and knowledge of which modules is already imported obviously
- exposing clause awareness - When at import MyModule exposing (<X>) it will only suggest candidates that are actually exposed from MyModule. It also filters out candidates already part of the exposing clause. This even works one level deeper for ADTs ie import MyModule exposing (MyADT(<X>)).
- When inside a function body, in addition to module level symbols and imported candidate symbols and param names, record fields, destructured params and destructuring aliases are used to supply relevant completion candidates.
Test support - Support for running a single elm test, all tests in a module or all tests in a project is supported by checking all functions that are annotated to return Test and builds a test suite on the fly to by run by the Elm node-test-runner.
Interactive Module diagram - D3 diagram to show module dependencies (grouped by folder) within your project, where you can filter by directory and select to display varying degrees of details. Requires knowledge of all modules, how they are related and what's exposed from them.
Highlight exposeds All top level symbols that are exposed from a module have a marker in the gutter of the editor to make them stand out from module private stuff.
Simple refactor support
Quick import : by entering a wanted <alias>.<X> and invoking a command the editor will try to find candidates that satisfies <X> if it's only one candidate the appropriate module is added as an import with the given alias. If multiple candidate you are prompted to select one.
Quick expose/unexpose : With the cursor placed inside the range of a top level definition that may be exposed you can invoke a command to expose it. It will then be added to the exposing clause of the module definition (unexpose works the reverse obviously)
Quickfixes - There is a linter feature that invokes elm-make with json output to highlight errors and warnings, some of these are fairly parseable so one can provide quick-fix shortcuts (like adding type annotation, correcting the type annotation, fixing field naming mistmatches etc.). This could be so so much better with a more machine friendly/structured format :-)
Package doc preview - View a preview of package docs for a Module from within the editor (uses elm executable to create doc json file and renders markdown)
Package manager - Wrapper around elm-package.json and exact-dependencies.json that allows you to view and manage your package dependencies quickly without leaving the editor
Package dependency diagram - In editor somewhat interactive D3 DAG that shows direct and transitive dependencies with their resolved version numbers and package description. Uses the all-packages endpoint from the package site
elm-reactor - Start reactor and view stuff quickly using the in built chromium browser in Light Table
elm-repl - Use any Elm "buffer" in Light Table to evaluate statements and view results inline next to the statement you evaluate. Super for explorative stuff.
elm-format - Format buffer, saved file, current top level definition using elm format. Typically hooked to the save command in light table (together with the linting feature)

My top feature wishlist

Extract function : I would love to be able to select an expression and extract that to a separate function, and have it automatically get the correct parameters defined and the appropriate type annotation. I think it's hard/(impossible at times?) to do this reliably without (proper) type inference... the kind the elm compiler works out.
Infer type/type holes : elm-jutsu has some of this, but I don't particularily want to go down the same (me assuming here) implementation route of calling elm-make and parsing text from standard out/std err.
Rock solid rename refactor support : This is something everyone wants and I would think most people have come to expect from their editor (atleast IDEs). Making a half arsed 80% rename in a large project is no fun at all. Worse even when you end up renaming symbols that shouldn't have been. A 100 (99%) solution for find usages from a tool, would give everthing an editor would need to peform a rename, alternatively one could choose to let the tool do it for you (but then you have the problem of unsaved buffers...)

Other stuff for anyone still reading :)

Implementation overview

Elm light consists of a node server and a node client (the editor). The server is started when you connect to an Elm project (details on how is left out for brevity). When connecting the server performs an initial elm-package install to ensure you have all 3rd party packages installed locally. It then parses all the project sources and all 3rd party package sources (by parsing your projects elm-package.json and parsing 3.rd party packages elm-package.json to ensure it parses only Elm files exposed by these 3rd party packages.). The server also startes another server process Chokidar which listens for changes to all relevant directories and reparses as necessary (typically when you save a file in your editor, but also if you install or uninstall packages). The result from parsing is file AST's represented as JSON which are streamed to the editor/client using Node's IPC protocol. The editor keeps hold of all the file AST's in a ClojureScript Atom.

When content changes in an Elm editor, the whole file is reparsed (limited by a debounce). Only successful parses are stored in the client atom.

Parsing

The parser is implemented by using a parser generator (PEG.js). It's ok for small files, but doesn't perform particularily well on larger files (7000 lines takes roughly 250 ms on my machine). The parser only parses top level definitions (meaning it doesn't parse function bodies). Anyways the bottom line is that I have access to:

Doc comments
module definition declaration with exposing clause
import declarations (with alias and exposing clause)
type declarations
type aliases
function/value annotations
function/value "headers" (params, including destructuring, aliases etc) every AST node contains location information (to - from)

Ok enough with the intro stuff. What features are currently supported

Sources of inspiration for a better elm-oracle

Language server protocol (open, evolved from VS Code).
Rust Language Server
Purescript IDE

I'm not saying it has to be a server, but if we are going to do something a bit more ambitious it feels like an option that should be considered. An executable might be simpler, but would it be able to provide the same level of support a language server could ?

Let's make some assumptions (they might be flawed, but play along anyways);

Parsing everything everytime wouldn't provide acceptable responsetimes. Caching of some sorts would probably be needed to support advanced features.
Very often when you need the tooling help the current Elm file is very likely not in a valid state, so parsing has to be forgiving and/or have knowledge of last valid state. Perhaps ?
Maybe it would be possible to "stream" info about editor changes to the tool (including current cursor position) and work out magic from there.
Files relevant to your project can change in various ways (deleted, modified in another tool, package install from the command line etc etc), maybe it makes sense to have the tool be a daemon to help keep track of those changes and update it's knowledge accordingly ?

Would Editors be able to support something like the Language Server Protocol (or simpler if that makes more sense). Well VS Code, Atom and Light Table certainly can. I'd be surprised if it wasn't possible in the other big mainstream editors too, but more research would be needed.

Worth checking out this community page too: http://langserver.org/

How does this last point tie into GSOC ?

Maybe it doesn't, but maybe it could too. I mean the first step could be a prototype/spike with just one feature that we test from Atom and Light Table (and maybe emacs/vim if someone with the knowhow and time steps up) to gain some experience. Just a thought.

Should it be of interest, even if only for a quick prototype, the node server part of elm-light including/or the peg.js grammar is of course possible to use as a starting point.

rundis/elm-light-features.md