D: start documenting stuff

Cargo.toml

@@ -2,6 +2,7 @@
 name = "libsyntax2"
 version = "0.1.0"
 authors = ["Aleksey Kladov <aleksey.kladov@gmail.com>"]
+license = "MIT OR Apache-2.0"
 
 [dependencies]
 unicode-xid = "0.1.0"

README.md

@@ -0,0 +1,5 @@
+# libsyntax2.0
+
+libsyntax2.0 is an **experimental** implementation of the corresponding [RFC](https://github.com/rust-lang/rfcs/pull/2256).
+
+See `docs` folder for more details.

docs/ARCHITECTURE.md

@@ -0,0 +1 @@
+# Design and open questions about libsyntax.

docs/CONTRIBUTING.md

@@ -0,0 +1,32 @@
+The project is in its early stages: contributions are welcome and
+would be **very** helpful, but the project is not *yet* optimized for
+contributors. Moreover, it is doubly experimental, so there's no
+guarantee that any work here would reach production. That said, here
+are some arias where contributions would be **especially** welcome:
+
+
+* Designing internal data structures: RFC only outlines the
+  constraints, it's an open question how to satisfy them in the
+  optimal way. See `ARCHITECTURE.md` for current design questions.
+  
+* Porting libsyntax parser to libsyntax2: currently libsyntax2 parses
+  only a tiny subset of Rust. This should be fixed by porting parsing
+  functions from libsyntax one by one.
+  
+* Writing validators: by design, libsyntax2 is very lax about the
+  input. For example, the lexer happily accepts unclosed strings. The
+  idea is that there should be a higher level visitor, which walks the
+  syntax tree after parsing and produces all the warnings. Alas,
+  there's no such visitor yet :( Would you like to write one? :)
+  
+* Creating tests: it would be tremendously helpful to read each of
+  libsyntax and libsyntax2 parser functions and crate a small separate
+  test cases to cover each and every edge case.
+  
+* Building stuff with libsyntax2: it would be really cool to compile
+  libsyntax2 to WASM and add *client side* syntax validation to rust
+  playground!
+
+
+Do take a look at the issue tracker, and try to read other docs in
+this folder.

docs/TESTS.md

@@ -0,0 +1,30 @@
+# libsyntax2.0 testing infrastructure
+
+Libsyntax2.0 tests are in the `tests/data` directory. Each test is a
+pair of files, an `.rs` file with Rust code and a `.txt` file with a
+human-readable representation of syntax tree.
+
+The test suite is intended to be independent from a particular parser:
+that's why it is just a list of files.
+
+The test suite is intended to be progressive: that is, if you want to
+write a Rust parser, you can TDD it by working through the test in
+order. That's why each test file begins with the number. Generally,
+tests should be added in order of the appearance of corresponding
+functionality in libsytnax2.0. If a bug in parser is uncovered, a
+**new** test should be created instead of modifying an existing one:
+it is preferable to have a gazillion of small isolated test files,
+rather than a single file which covers all edge cases. It's okay for
+files to have the same name except for the leading number. In general,
+test suite should be append-only: old tests should not be modified,
+new tests should be created instead.
+
+
+Note that only `ok` tests are normative: `err` tests test error
+recovery and it is totally ok for a parser to not implement any error
+recovery at all. However, for libsyntax2.0 we do care about error
+recovery, and we do care about precise and useful error messages.
+
+
+Contribution opportunity: design and implement testing infrastructure
+for validators.