Mirrors/rust-cookbook
2
0
Fork 0
mirror of https://github.com/rust-lang-nursery/rust-cookbook synced 2024-11-21 19:13:07 +00:00
Code Issues Projects Releases Packages Wiki Activity

Restructure Cookbook (#404)

Browse source
This commit is contained in:
Andrew Gauger 2018-07-10 14:07:24 -07:00 committed by GitHub
parent 601873ec69
commit 97dabe59ae
No known key found for this signature in database
GPG key ID: 4AEE18F83AFDEB23
175 changed files with 6550 additions and 6692 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

2
.github/PULL_REQUEST_TEMPLATE.md vendored
View file

@ -12,6 +12,8 @@ No worries if anything in these lists is unclear. Just submit the PR and ask awa
- [ ] commits are squashed into one and rebased to latest master
- [ ] PR contains correct "fixes #ISSUE_ID" clause to autoclose the issue on PR merge
- if issue does not exist consider creating it or remove the clause
- [ ] spell check runs without errors `./ci/spellchecker.sh`
- [ ] link check runs without errors `link-checker ./book`
- [ ] non rendered items are in sorted order (links, reference, identifiers, Cargo.toml)
- [ ] links to docs.rs have wildcard version `https://docs.rs/tar/*/tar/struct.Entry.html`
- [ ] example has standard [error handling](https://rust-lang-nursery.github.io/rust-cookbook/about.html#a-note-about-error-handling)

82
CONTRIBUTING.md
View file

@ -44,16 +44,68 @@ To run the cookbook test suite:
cargo test
```
## Linters
The Rust Cookbook comes with link checking and spell checking linters that
run on the continuous integration server. These linters should be run locally
before submitting a pull request to ensure there are no dead links or spelling
errors made.
To install the link checker, review the documentation for [python] to install
python 3.6 and pip3. Installing link-checker once the dependencies are met
is done with pip3.
```
[sudo] pip3 install link-checker==0.1.0
```
Alternatively, set up the user install directory on your PATH variable and
install link-checker for your user
```
pip3 install -user link-checker==0.1.0
```
Checking the links of the book locally first requires the book to be built
with mdBook. From the root directory of the cookbook, the following commands
run the link checker.
```
mdbook build
link-checker ./book
```
The aspell binary provides spell checking. Apt packages provide installation
on Debian based operating systems.
```
[sudo] apt install aspell -y
```
To check the spelling of the Rust Cookbook locally, run the following command
from the root of the Cookbook.
```
./ci/spellchecker.sh
```
If the spell checker finds a misspelled word, you have the opportunity to
correct the spelling mistake with the number keys. If the spelling mistake
is erroneous, add the word to the dictionary located in `ci/dictionary.txt`.
Pressing `a` or `l` will not add the word to the custom dictionary.
[mdbook]: http://azerupi.github.io/mdBook/index.html
[python]: https://packaging.python.org/tutorials/installing-packages/#install-pip-setuptools-and-wheel
[skeptic]: https://github.com/brson/rust-skeptic
## Finding what to contribute
This project is intended to be simple to contribute to, and to always
have obvious next work items available. If at any time there is not
something obvious to contribute, that is a bug. Please ask for
assistance on the [libz blitz] thread, or email Brian Anderson
directly (banderson@mozilla.com).
something obvious to contribute, that is a bug. Feel free to ask for
additional support at the
[Rust Ecosystem Working Group](https://gitter.im/rust-lang/WG-ecosystem).
The development process for the cookbook is presently oriented around
crates: we decide which crates to represent in the cookbook, then come
@ -146,16 +198,18 @@ something a typical Rust user typically wants to do.
#### Description
Describe traits imported and the methods used. Think about what information
supports the use case and might not be obvious to someone new. Keep the
description to 1-4 sentences, avoiding explanations outside the scope of the
supports the use case and might not be obvious to someone new. Keep the
description to 1-4 sentences, avoiding explanations outside the scope of the
code sample.
Use third person narative of the code execution, taking the opportunity
to link to API documentation. Hyperlink all references to APIs, either
Use third person narrative of the code execution, taking the opportunity
to link to API documentation. Always use
[active voice](https://www.plainlanguage.gov/guidelines/conversational/use-active-voice/).
Hyperlink all references to APIs, either
on doc.rust-lang.org/std or docs.rs, and style them as `code`. Use
wildcard version specifiers for crate links.
Any requirements to execute the code that are not apparent, such as
Any requirements to execute the code that are not apparent, such as
passing environment flags, or configuring `Cargo.toml` should be added
after the code sample.
@ -164,8 +218,8 @@ after the code sample.
> distribution, then sample from that distribution using
> [`Distribution::sample`] with help of a random-number
> generator [`rand::Rng`].
>
> The [distributions available are documented here][rand-distributions].
>
> The [distributions available are documented here][rand-distributions].
> An example using the [`Normal`] distribution is shown below.
[uniform distribution]: https://en.wikipedia.org/wiki/Uniform_distribution_(continuous)
@ -190,11 +244,11 @@ error handling correctly and propagate errors with `?` (not `try!`, `urwrap`, or
`expect`). If there is no need for error handling in the example, prefer `main()`.
Avoid glob imports (`*`), even for preludes, so that users can see what
traits are called. (Some crates might consider using glob imports for preludes
traits are called. (Some crates might consider using glob imports for preludes
best practice, making this awkward.)
Examples should be simple and obvious enough that an experienced dev
do not need comments.
do not need comments.
Examples should compile without warnings, clippy lint warnings, or panics.
The code should be formatted by rustfmt. Hide all error boilerplate and
@ -206,9 +260,9 @@ explanation in the description.
> ```rust
> extern crate rand;
>
>
> use rand::distributions::{Normal, Distribution};
>
>
> fn main() {
> let mut rng = rand::thread_rng();
> let normal = Normal::new(2.0, 3.0);

2
Cargo.toml
View file

@ -54,6 +54,8 @@ syslog = "4.0"
[build-dependencies]
skeptic = "0.13"
walkdir = "2.0"
[dev-dependencies]
skeptic = "0.13"
walkdir = "2.0"

5
build.rs
View file

@ -1,9 +1,10 @@
extern crate skeptic;
extern crate walkdir;
use std::fs;
use walkdir::WalkDir;
fn main() {
let paths = fs::read_dir("./src/").unwrap()
let paths = WalkDir::new("./src/").into_iter()
// convert paths to Strings
.map(|p| p.unwrap().path().to_str().unwrap().to_string())
// only compile markdown files

6
ci/dictionary.txt
View file

@ -12,8 +12,11 @@ akshat
alisha
AlphaNumeric
ApiResponse
apis
APIs
Appender
appender
applicationx
args
ascii
ashley
@ -280,6 +283,7 @@ SystemRandom
SystemTime
SystemTimeError
TcpListener
tcpip
TcpStream
TempDir
tempdir
@ -297,6 +301,7 @@ tuple
Tuple
typesafe
unary
unix
unwinded
UpperHex
uptime
@ -310,6 +315,7 @@ usize
VarError
versa
Versioning
versioning
VersionReq
vreq
WalkDir

56
src/SUMMARY.md
View file

@ -2,11 +2,53 @@
[Table of Contents](intro.md)
[About](about.md)
- [Basics](basics.md)
- [Encoding](encoding.md)
- [Algorithms](algorithms.md)
- [Generate Random Values](algorithms/randomness.md)
- [Command Line](cli.md)
- [Argument Parsing](cli/arguments.md)
- [Compression](compression.md)
- [Working with Tarballs](compression/tar.md)
- [Concurrency](concurrency.md)
- [Networking](net.md)
- [Application development](app.md)
- [Logging](logging.md)
- [Build Time Tooling](build_tools.md)
- [Explicit Threads](concurrency/threads.md)
- [Data Parallelism](concurrency/parallel.md)
- [Cryptography](cryptography.md)
- [Hashing](cryptography/hashing.md)
- [Encryption](cryptography/encryption.md)
- [Data Structures](data_structures.md)
- [Bitfield](data_structures/bitfield.md)
- [Date and Time](datetime.md)
- [Duration and Calculation](datetime/duration.md)
- [Parsing and Displaying](datetime/parse.md)
- [Development Tools](development_tools.md)
- [Debugging](development_tools/debugging.md)
- [Log Messages](development_tools/debugging/log.md)
- [Configure Logging](development_tools/debugging/config_log.md)
- [Versioning](development_tools/versioning.md)
- [Build Time Tooling](development_tools/build_tools.md)
- [Encoding](encoding.md)
- [Character Sets](encoding/strings.md)
- [CSV processing](encoding/csv.md)
- [Structured Data](encoding/complex.md)
- [Error Handling](errors.md)
- [Handle Error Variants](errors/handle.md)
- [File System](file.md)
- [Read & Write](file/read-write.md)
- [Directory Traversal](file/dir.md)
- [Hardware Support](hardware.md)
- [Processor](hardware/processor.md)
- [Memory Management](mem.md)
- [Global Static](mem/global_static.md)
- [Network](net.md)
- [Server](net/server.md)
- [Operating System](os.md)
- [External Command](os/external.md)
- [Text Processing](text.md)
- [Regular Expressions](text/regex.md)
- [Web Programming](web.md)
- [Extracting Links](web/scraping.md)
- [URL](web/url.md)
- [Media Types](web/mime.md)
- [Clients](web/clients.md)
- [Making Requests](web/clients/requests.md)
- [Calling a Web API](web/clients/apis.md)
- [Downloads](web/clients/download.md)

19
src/algorithms.md Normal file
View file

@ -0,0 +1,19 @@
# Algorithms
| Recipe | Crates | Categories |
|--------|--------|------------|
| [Generate random numbers][ex-rand] | [![rand-badge]][rand] | [![cat-science-badge]][cat-science] |
| [Generate random numbers within a range][ex-rand-range] | [![rand-badge]][rand] | [![cat-science-badge]][cat-science] |
| [Generate random numbers with given distribution][ex-rand-dist] | [![rand-badge]][rand] | [![cat-science-badge]][cat-science] |
| [Generate random values of a custom type][ex-rand-custom] | [![rand-badge]][rand] | [![cat-science-badge]][cat-science] |
| [Create random passwords from a set of alphanumeric characters][ex-rand-passwd] | [![rand-badge]][rand] | [![cat-os-badge]][cat-os] |
| [Create random passwords from a set of user-defined characters][ex-rand-choose] | [![rand-badge]][rand] | [![cat-os-badge]][cat-os] |
[ex-rand]: algorithms/randomness.html#generate-random-numbers
[ex-rand-range]: algorithms/randomness.html#generate-random-numbers-within-a-range
[ex-rand-dist]: algorithms/randomness.html#generate-random-numbers-with-given-distribution
[ex-rand-custom]: algorithms/randomness.html#generate-random-values-of-a-custom-type
[ex-rand-passwd]: algorithms/randomness.html#create-random-passwords-from-a-set-of-alphanumeric-characters
[ex-rand-choose]: algorithms/randomness.html#create-random-passwords-from-a-set-of-user-defined-characters
{{#include links.md}}

15
src/algorithms/randomness.md Normal file
View file

@ -0,0 +1,15 @@
# Generate Random Values
{{#include randomness/rand.md}}
{{#include randomness/rand-range.md}}
{{#include randomness/rand-dist.md}}
{{#include randomness/rand-custom.md}}
{{#include randomness/rand-passwd.md}}
{{#include randomness/rand-choose.md}}
{{#include ../links.md}}

26
src/algorithms/randomness/rand-choose.md Normal file
View file

@ -0,0 +1,26 @@
## Create random passwords from a set of user-defined characters
[![rand-badge]][rand] [![cat-os-badge]][cat-os]
Randomly generates a string of given length ASCII characters with custom user-defined bytestring, with [`choose`].
```rust
extern crate rand;
use rand::{thread_rng, Rng};
fn main() {
const CHARSET: &[u8] = b"ABCDEFGHIJKLMNOPQRSTUVWXYZ\
abcdefghijklmnopqrstuvwxyz\
0123456789)(*&^%$#@!~";
let mut rng = thread_rng();
let password: Option<String> = (0..30)
.map(|_| Some(*rng.choose(CHARSET)? as char))
.collect();
println!("{:?}", password);
}
```
[`choose`]: https://docs.rs/rand/*/rand/trait.Rng.html#method.choose

40
src/algorithms/randomness/rand-custom.md Normal file
View file

@ -0,0 +1,40 @@
## Generate random values of a custom type
[![rand-badge]][rand] [![cat-science-badge]][cat-science]
Randomly generates a tuple `(i32, bool, f64)` and variable of user defined type `Point`.
Implements the [`Distribution`] trait on type Point for [`Standard`] in order to allow random generation.
```rust
extern crate rand;
use rand::Rng;
use rand::distributions::{Distribution, Standard};
#[derive(Debug)]
struct Point {
x: i32,
y: i32,
}
impl Distribution<Point> for Standard {
fn sample<R: Rng + ?Sized>(&self, rng: &mut R) -> Point {
let (rand_x, rand_y) = rng.gen();
Point {
x: rand_x,
y: rand_y,
}
}
}
fn main() {
let mut rng = rand::thread_rng();
let rand_tuple = rng.gen::<(i32, bool, f64)>();
let rand_point: Point = rng.gen();
println!("Random tuple: {:?}", rand_tuple);
println!("Random Point: {:?}", rand_point);
}
```
[`Distribution`]: https://docs.rs/rand/*/rand/distributions/trait.Distribution.html
[`Standard`]: https://docs.rs/rand/*/rand/distributions/struct.Standard.html

32
src/algorithms/randomness/rand-dist.md Normal file
View file

@ -0,0 +1,32 @@
## Generate random numbers with given distribution
[![rand-badge]][rand] [![cat-science-badge]][cat-science]
By default, random numbers have [uniform distribution].
To generate numbers with other distributions you instantiate a
distribution, then sample from that distribution using
[`IndependentSample::ind_sample`] with help of a random-number
generator [`rand::Rng`].
The [distributions available are documented here][rand-distributions]. An example using the
[`Normal`] distribution is shown below.
```rust
extern crate rand;
use rand::distributions::{Normal, Distribution};
fn main() {
let mut rng = rand::thread_rng();
let normal = Normal::new(2.0, 3.0);
let v = normal.sample(&mut rng);
println!("{} is from a N(2, 9) distribution", v)
}
```
[`Distribution::sample`]: https://docs.rs/rand/*/rand/distributions/trait.Distribution.html#tymethod.sample
[`Normal`]: https://docs.rs/rand/*/rand/distributions/normal/struct.Normal.html
[`rand::Rng`]: https://docs.rs/rand/*/rand/trait.Rng.html
[rand-distributions]: https://docs.rs/rand/*/rand/distributions/index.html
[uniform distribution]: https://en.wikipedia.org/wiki/Uniform_distribution_(continuous)

24
src/algorithms/randomness/rand-passwd.md Normal file
View file

@ -0,0 +1,24 @@
## Create random passwords from a set of alphanumeric characters
[![rand-badge]][rand] [![cat-os-badge]][cat-os]
Randomly generates a string of given length ASCII characters in the range `A-Z,
a-z, 0-9`, with [`Alphanumeric`] sample.
```rust
extern crate rand;
use rand::{thread_rng, Rng};
use rand::distributions::Alphanumeric;
fn main() {
let rand_string: String = thread_rng()
.sample_iter(&Alphanumeric)
.take(30)
.collect();
println!("{}", rand_string);
}
```
[`Alphanumeric`]: https://docs.rs/rand/*/rand/distributions/struct.Alphanumeric.html

44
src/algorithms/randomness/rand-range.md Normal file
View file

@ -0,0 +1,44 @@
## Generate random numbers within a range
[![rand-badge]][rand] [![cat-science-badge]][cat-science]
Generates a random value within half-open `[0, 10)` range (not including `10`) with [`Rng::gen_range`].
```rust
extern crate rand;
use rand::Rng;
fn main() {
let mut rng = rand::thread_rng();
println!("Integer: {}", rng.gen_range(0, 10));
println!("Float: {}", rng.gen_range(0.0, 10.0));
}
```
[`Range`] can obtain values with [uniform distribution].
This has the same effect, but may be faster when repeatedly generating numbers
in the same range.
```rust
extern crate rand;
use rand::distributions::{Range, Distribution};
fn main() {
let mut rng = rand::thread_rng();
let die = Range::new(1, 7);
loop {
let throw = die.sample(&mut rng);
println!("Roll the die: {}", throw);
if throw == 6 {
break;
}
}
}
```
[`Uniform`]: https://docs.rs/rand/*/rand/distributions/uniform/struct.Uniform.html
[`Rng::gen_range`]: https://doc.rust-lang.org/rand/*/rand/trait.Rng.html#method.gen_range
[uniform distribution]: https://en.wikipedia.org/wiki/Uniform_distribution_(continuous)

30
src/algorithms/randomness/rand.md Normal file
View file

@ -0,0 +1,30 @@
## Generate random numbers
[![rand-badge]][rand] [![cat-science-badge]][cat-science]
Generates random numbers with help of random-number
generator [`rand::Rng`] obtained via [`rand::thread_rng`]. Each thread has an
intialized generator. Integers are uniformly distributed over the range of the
type, and floating point numbers are uniformly distributed from 0 up to but not
including 1.
```rust
extern crate rand;
use rand::Rng;
fn main() {
let mut rng = rand::thread_rng();
let n1: u8 = rng.gen();
let n2: u16 = rng.gen();
println!("Random u8: {}", n1);
println!("Random u16: {}", n2);
println!("Random u32: {}", rng.gen::<u32>());
println!("Random i32: {}", rng.gen::<i32>());
println!("Random float: {}", rng.gen::<f64>());
}
```
[`rand::Rng`]: https://docs.rs/rand/*/rand/trait.Rng.html
[`rand::thread_rng`]: https://docs.rs/rand/*/rand/fn.thread_rng.html

882
src/app.md
View file

@ -1,882 +0,0 @@
# Application development
| Recipe | Crates | Categories |
|--------|--------|------------|
| [Parse command line arguments][ex-clap-basic] | [![clap-badge]][clap] | [![cat-command-line-badge]][cat-command-line] |
| [Decompress a tarball][ex-tar-decompress] | [![flate2-badge]][flate2] [![tar-badge]][tar] | [![cat-compression-badge]][cat-compression] |
| [Compress a directory into a tarball][ex-tar-compress] | [![flate2-badge]][flate2] [![tar-badge]][tar] | [![cat-compression-badge]][cat-compression] |
| [Decompress a tarball while removing a prefix from the paths][ex-tar-strip-prefix] | [![flate2-badge]][flate2] [![tar-badge]][tar] | [![cat-compression-badge]][cat-compression] |
| [Avoid writing and reading from a same file][ex-avoid-read-write] | [![same_file-badge]][same_file] | [![cat-filesystem-badge]][cat-filesystem] |
| [Find loops for a given path][ex-find-file-loops] | [![same_file-badge]][same_file] | [![cat-filesystem-badge]][cat-filesystem] |
| [Recursively find duplicate file names][ex-dedup-filenames] | [![walkdir-badge]][walkdir] | [![cat-filesystem-badge]][cat-filesystem] |
| [Recursively find all files with given predicate][ex-file-predicate] | [![walkdir-badge]][walkdir] | [![cat-filesystem-badge]][cat-filesystem] |
| [Traverse directories while skipping dotfiles][ex-file-skip-dot] | [![walkdir-badge]][walkdir] | [![cat-filesystem-badge]][cat-filesystem] |
| [Recursively calculate file sizes at given depth][ex-file-sizes] | [![walkdir-badge]][walkdir] | [![cat-filesystem-badge]][cat-filesystem] |
| [Find all png files recursively][ex-glob-recursive] | [![glob-badge]][glob] | [![cat-filesystem-badge]][cat-filesystem] |
| [Find all files with given pattern ignoring filename case][ex-glob-with] | [![glob-badge]][glob] | [![cat-filesystem-badge]][cat-filesystem] |
| [Parse and increment a version string][ex-semver-increment] | [![semver-badge]][semver] | [![cat-config-badge]][cat-config] |
| [Parse a complex version string][ex-semver-complex] | [![semver-badge]][semver] | [![cat-config-badge]][cat-config] |
| [Check if given version is pre-release][ex-semver-prerelease] | [![semver-badge]][semver] | [![cat-config-badge]][cat-config] |
| [Find the latest version satisfying given range][ex-semver-latest] | [![semver-badge]][semver] | [![cat-config-badge]][cat-config] |
| [Check external command version for compatibility][ex-semver-command] | [![semver-badge]][semver] | [![cat-text-processing-badge]][cat-text-processing] [![cat-os-badge]][cat-os]
[ex-clap-basic]: #ex-clap-basic
<a name="ex-clap-basic"></a>
## Parse command line arguments
[![clap-badge]][clap] [![cat-command-line-badge]][cat-command-line]
```rust
extern crate clap;
use clap::{Arg, App};
fn main() {
// Define command line arguments.
let matches = App::new("My Test Program")
.version("0.1.0")
.author("Hackerman Jones <hckrmnjones@hack.gov>")
.about("Teaches argument parsing")
.arg(Arg::with_name("file")
.short("f")
.long("file")
.takes_value(true)
.help("A cool file"))
.arg(Arg::with_name("num")
.short("n")
.long("number")
.takes_value(true)
.help("Five less than your favorite number"))
.get_matches();
// Get value for file, or default to 'input.txt'.
let myfile = matches.value_of("file").unwrap_or("input.txt");
println!("The file passed is: {}", myfile);
// Get value for num if present, and try parsing it as i32.
let num_str = matches.value_of("num");
match num_str {
None => println!("No idea what your favorite number is."),
Some(s) => {
match s.parse::<i32>() {
Ok(n) => println!("Your favorite number must be {}.", n + 5),
Err(_) => println!("That's not a number! {}", s),
}
}
}
}
```
The `clap` crate is a simple-to-use, efficient, and full-featured library for
parsing command line arguments and subcommands when writing console/terminal
applications.
The application can describe the structure of its command-line interface using
`clap`'s builder style. The [documentation] gives two other possible ways to
instantiate an application.
[documentation]: https://docs.rs/clap/
In the builder style, `with_name` is the unique identifier that `value_of` will
use to retrieve the value passed. The `short` and `long` options control the
flag the user will be expected to type; short flags look like `-f` and long
flags look like `--file`.
Usage information is generated by `clap`. The usage for the example application
looks like this.
```
My Test Program 0.1.0
Hackerman Jones <hckrmnjones@hack.gov>
Teaches argument parsing
USAGE:
testing [OPTIONS]
FLAGS:
-h, --help Prints help information
-V, --version Prints version information
OPTIONS:
-f, --file <file> A cool file
-n, --number <num> Five less than your favorite number
```
We can test the application by running a command like the following.
```
$ cargo run -- -f myfile.txt -n 251
```
The output is:
```
The file passed is: myfile.txt
Your favorite number must be 256.
```
[ex-tar-decompress]: #ex-tar-decompress
<a name="ex-tar-decompress"></a>
## Decompress a tarball
[![flate2-badge]][flate2] [![tar-badge]][tar] [![cat-compression-badge]][cat-compression]
Decompress ([`GzDecoder`]) and
extract ([`Archive::unpack`]) all files from a compressed tarball
named `archive.tar.gz` located in the current working directory.
```rust,no_run
# #[macro_use]
# extern crate error_chain;
extern crate flate2;
extern crate tar;
use std::fs::File;
use flate2::read::GzDecoder;
use tar::Archive;
#
# error_chain! {
# foreign_links {
# Io(std::io::Error);
# }
# }
fn run() -> Result<()> {
let path = "archive.tar.gz";
// Open a compressed tarball
let tar_gz = File::open(path)?;
// Decompress it
let tar = GzDecoder::new(tar_gz);
// Load the archive from the tarball
let mut archive = Archive::new(tar);
// Unpack the archive inside current working directory
archive.unpack(".")?;
Ok(())
}
#
# quick_main!(run);
```
[ex-tar-compress]: #ex-tar-compress
<a name="ex-tar-compress"></a>
## Compress a directory into tarball
[![flate2-badge]][flate2] [![tar-badge]][tar] [![cat-compression-badge]][cat-compression]
Compresses `/var/log` directory into `archive.tar.gz`.
Creates a [`File`] wrapped in [`GzEncoder`]
and [`tar::Builder`]. </br>Adds contents of `/var/log` directory recursively into the archive
under `backup/logs`path with [`Builder::append_dir_all`].
[`GzEncoder`] is responsible for transparently compressing the
data prior to writing it into `archive.tar.gz`.
```rust,no_run
# #[macro_use]
# extern crate error_chain;
extern crate tar;
extern crate flate2;
#
# error_chain! {
# foreign_links {
# Io(std::io::Error);
# }
# }
use std::fs::File;
use flate2::Compression;
use flate2::write::GzEncoder;
fn run() -> Result<()> {
let tar_gz = File::create("archive.tar.gz")?;
let enc = GzEncoder::new(tar_gz, Compression::default());
let mut tar = tar::Builder::new(enc);
tar.append_dir_all("backup/logs", "/var/log")?;
Ok(())
}
#
# quick_main!(run);
```
[ex-tar-strip-prefix]: #ex-tar-strip-prefix
<a name="ex-tar-strip-prefix"></a>
## Decompress a tarball while removing a prefix from the paths
[![flate2-badge]][flate2] [![tar-badge]][tar] [![cat-compression-badge]][cat-compression]
Strip a path prefix from the entries of a tarball before unpacking them.
We iterate over the [`Archive::entries`], using [`Path::strip_prefix`] to remove
the specified path prefix (`bundle/logs`) before extracting the [`tar::Entry`]
via [`Entry::unpack`].
```rust,no_run
# #[macro_use]
# extern crate error_chain;
extern crate flate2;
extern crate tar;
#
# error_chain! {
# foreign_links {
# Io(std::io::Error);
# StripPrefixError(::std::path::StripPrefixError);
# }
# }
use std::fs::File;
use std::path::PathBuf;
use flate2::read::GzDecoder;
use tar::Archive;
fn run() -> Result<()> {
let file = File::open("archive.tar.gz")?;
let mut archive = Archive::new(GzDecoder::new(file));
let prefix = "bundle/logs";
println!("Extracted the following files:");
archive
.entries()?
.filter_map(|e| e.ok())
.map(|mut entry| -> Result<PathBuf> {
// Need to get owned data to break the borrow loop
let path = entry.path()?.strip_prefix(prefix)?.to_owned();
entry.unpack(&path)?;
Ok(path)
})
.filter_map(|e| e.ok())
.for_each(|x| println!("> {}", x.display()));
Ok(())
}
#
# quick_main!(run);
```
[ex-avoid-read-write]: #ex-avoid-read-write
<a name="ex-avoid-read-write"></a>
## Avoid writing and reading from a same file
[![same_file-badge]][same_file] [![cat-filesystem-badge]][cat-filesystem]
Use [`same_file::Handle`] to a file that can be tested for equality with
other handles. In this example, the handles of file to be read from and
to be written to are tested for equality.
```bash
cargo run
```
will display the contents of the file if the two files are not same and
```bash
cargo run >> ./new.txt
```
will display the error (because the two files are same).
```rust,no_run
# #[macro_use]
# extern crate error_chain;
extern crate same_file;
use same_file::Handle;
use std::path::Path;
use std::fs::File;
use std::io::{BufRead, BufReader};
#
# error_chain! {
# foreign_links {
# IOError(::std::io::Error);
# }
# }
fn run() -> Result<()> {
let path_to_read = Path::new("new.txt");
let stdout_handle = Handle::stdout()?;
let handle = Handle::from_path(path_to_read)?;
if stdout_handle == handle {
bail!("You are reading and writing to the same file");
} else {
let file = File::open(&path_to_read)?;
let file = BufReader::new(file);
for (num, line) in file.lines().enumerate() {
println!("{} : {}", num, line?.to_uppercase());
}
}
Ok(())
}
#
# quick_main!(run);
```
[ex-find-file-loops]: #ex-find-file-loops
<a name="ex-find-file-loops"></a>
## Find loops for a given path
[![same_file-badge]][same_file] [![cat-filesystem-badge]][cat-filesystem]
Use [`same_file::is_same_file`] to detect loops for a given path.
For example, a loop could be created on a Unix system via symlinks:
```bash
mkdir -p /tmp/foo/bar/baz
ln -s /tmp/foo/ /tmp/foo/bar/baz/qux
```
Then, running the following would assert that there exists a loop.
```rust,no_run
extern crate same_file;
use std::io;
use std::path::{Path, PathBuf};
use same_file::is_same_file;
// Check this path against all of its parents
fn contains_loop<P: AsRef<Path>>(path: P) -> io::Result<Option<(PathBuf, PathBuf)>> {
let path = path.as_ref();
let mut path_buf = path.to_path_buf();
while path_buf.pop() {
if is_same_file(&path_buf, path)? {
return Ok(Some((path_buf, path.to_path_buf())));
} else if let Some(looped_paths) = contains_loop(&path_buf)? {
return Ok(Some(looped_paths));
}
}
return Ok(None);
}
fn main() {
assert_eq!(
contains_loop("/tmp/foo/bar/baz/qux/bar/baz").unwrap(),
Some((
PathBuf::from("/tmp/foo"),
PathBuf::from("/tmp/foo/bar/baz/qux")
))
);
}
```
[ex-dedup-filenames]: #ex-dedup-filenames
<a name="ex-dedup-filenames"></a>
## Recursively find duplicate file names
[![walkdir-badge]][walkdir] [![cat-filesystem-badge]][cat-filesystem]
Find recursively in the current directory duplicate filenames,
printing them only once.
```rust,no_run
extern crate walkdir;
use std::collections::HashMap;
use walkdir::WalkDir;
fn main() {
// Counters indexed by filenames
let mut filenames = HashMap::new();
// List recusively all files in the current directory filtering out
// directories and files not accessible (permission denied)
for entry in WalkDir::new(".")
.into_iter()
.filter_map(Result::ok)
.filter(|e| !e.file_type().is_dir()) {
// Get entry's filename
let f_name = String::from(entry.file_name().to_string_lossy());
// Get or initialize the counter
let counter = filenames.entry(f_name.clone()).or_insert(0);
// Update the counter
*counter += 1;
if *counter == 2 {
println!("{}", f_name);
}
}
}
```
[ex-file-predicate]: #ex-file-predicate
<a name="ex-file-predicate"></a>
## Recursively find all files with given predicate
[![walkdir-badge]][walkdir] [![cat-filesystem-badge]][cat-filesystem]
Find JSON files modified within the last day in the current directory.
Using [`follow_links`] ensures symbolic links are followed like they were
normal directories and files.
```rust,no_run
# #[macro_use]
# extern crate error_chain;
extern crate walkdir;
use walkdir::WalkDir;
#
# error_chain! {
# foreign_links {
# WalkDir(walkdir::Error);
# Io(std::io::Error);
# SystemTime(std::time::SystemTimeError);
# }
# }
fn run() -> Result<()> {
// List recusively all accessible files in the current directory
for entry in WalkDir::new(".")
.follow_links(true)
.into_iter()
.filter_map(|e| e.ok()) {
// Get entry's filename
let f_name = entry.file_name().to_string_lossy();
// Get entry's modified time
let sec = entry.metadata()?.modified()?;
// Print JSON files modified within the last day
if f_name.ends_with(".json") && sec.elapsed()?.as_secs() < 86400 {
println!("{}", f_name);
}
}
Ok(())
}
#
# quick_main!(run);
```
[ex-file-skip-dot]: #ex-file-skip-dot
<a name="ex-file-skip-dot"></a>
## Traverse directories while skipping dotfiles
[![walkdir-badge]][walkdir] [![cat-filesystem-badge]][cat-filesystem]
Uses [`filter_entry`] to descend recursively into entries passing the `is_not_hidden` predicate thus skipping hidden files and directories whereas [`Iterator::filter`] would be applied to each [`WalkDir::DirEntry`] even if the parent is a hidden directory.
Root dir `"."` is yielded due to [`WalkDir::depth`] usage in `is_not_hidden` predicate.
```rust,no_run
extern crate walkdir;
use walkdir::{DirEntry, WalkDir};
fn is_not_hidden(entry: &DirEntry) -> bool {
entry
.file_name()
.to_str()
.map(|s| entry.depth() == 0 || !s.starts_with("."))
.unwrap_or(false)
}
fn main() {
WalkDir::new(".")
.into_iter()
.filter_entry(|e| is_not_hidden(e))
.filter_map(|v| v.ok())
.for_each(|x| println!("{}", x.path().display()));
}
```
[ex-file-sizes]: #ex-file-sizes
<a name="ex-file-sizes"></a>
## Recursively calculate file sizes at given depth
[![walkdir-badge]][walkdir] [![cat-filesystem-badge]][cat-filesystem]
Recursion depth can be flexibly set by [`WalkDir::min_depth`] & [`WalkDir::max_depth`] methods.
In this example we sum all file sizes to 3 subfolders depth, ignoring files in the root folder
at the same time.
```rust
extern crate walkdir;
use walkdir::WalkDir;
fn main() {
let total_size = WalkDir::new(".")
.min_depth(1)
.max_depth(3)
.into_iter()
.filter_map(|entry| entry.ok()) // Files, we have access to
.filter_map(|entry| entry.metadata().ok()) // Get metadata
.filter(|metadata| metadata.is_file()) // Filter out directories
.fold(0, |acc, m| acc + m.len()); // Accumulate sizes
println!("Total size: {} bytes.", total_size);
}
```
[ex-glob-recursive]: #ex-glob-recursive
<a name="ex-glob-recursive"></a>
## Find all png files recursively
[![glob-badge]][glob] [![cat-filesystem-badge]][cat-filesystem]
Recursively find all PNG files in the current directory.
In this case, the `**` pattern matches the current directory and all subdirectories.
You can also use the `**` pattern for any directory, not just the current one.
For example, `/media/**/*.png` would match all PNGs in `media` and it's subdirectories.
```rust,no_run
# #[macro_use]
# extern crate error_chain;
extern crate glob;
use glob::glob;
#
# error_chain! {
# foreign_links {
# Glob(glob::GlobError);
# Pattern(glob::PatternError);
# }
# }
fn run() -> Result<()> {
for entry in glob("**/*.png")? {
println!("{}", entry?.display());
}
Ok(())
}
#
# quick_main!(run);
```
[ex-glob-with]: #ex-glob-with
<a name="ex-glob-with"></a>
## Find all files with given pattern ignoring filename case.
[![glob-badge]][glob] [![cat-filesystem-badge]][cat-filesystem]
Find all image files in the `/media/` directory matching the `img_[0-9]*.png` pattern.
A custom [`MatchOptions`] struct is passed to the [`glob_with`] function making the glob pattern case insensitive while keeping the other options [`Default`].
```rust,no_run
# #[macro_use]
# extern crate error_chain;
extern crate glob;
use glob::{glob_with, MatchOptions};
#
# error_chain! {
# foreign_links {
# Glob(glob::GlobError);
# Pattern(glob::PatternError);
# }
# }
fn run() -> Result<()> {
let options = MatchOptions {
case_sensitive: false,
..Default::default()
};
for entry in glob_with("/media/img_[0-9]*.png", &options)? {
println!("{}", entry?.display());
}
Ok(())
}
#
# quick_main!(run);
```
[ex-semver-increment]: #ex-semver-increment
<a name="ex-semver-increment"></a>
## Parse and increment a version string.
[![semver-badge]][semver] [![cat-config-badge]][cat-config]
Constructs a [`semver::Version`] from a string literal using [`Version::parse`], then increments it by patch, minor, and major version number one by one.
Note that in accordance with the [Semantic Versioning Specification], incrementing the minor version number resets the patch version number to 0 and incrementing the major version number resets both the minor and patch version numbers to 0.
```rust
# #[macro_use]
# extern crate error_chain;
extern crate semver;
use semver::Version;
#
# error_chain! {
# foreign_links {
# SemVer(semver::SemVerError);
# }
# }
fn run() -> Result<()> {
let mut parsed_version = Version::parse("0.2.6")?;
assert_eq!(
parsed_version,
Version {
major: 0,
minor: 2,
patch: 6,
pre: vec![],
build: vec![],
}
);
parsed_version.increment_patch();
assert_eq!(parsed_version.to_string(), "0.2.7");
println!("New patch release: v{}", parsed_version);
parsed_version.increment_minor();
assert_eq!(parsed_version.to_string(), "0.3.0");
println!("New minor release: v{}", parsed_version);
parsed_version.increment_major();
assert_eq!(parsed_version.to_string(), "1.0.0");
println!("New major release: v{}", parsed_version);
Ok(())
}
#
# quick_main!(run);
```
[ex-semver-complex]: #ex-semver-complex
<a name="ex-semver-complex"></a>
## Parse a complex version string.
[![semver-badge]][semver] [![cat-config-badge]][cat-config]
Constructs a [`semver::Version`] from a complex version string using [`Version::parse`]. The string
contains pre-release and build metadata as defined in the [Semantic Versioning Specification].
Note that, in accordance with the Specification, build metadata is parsed but not considered when
comparing versions. In other words, two versions may be equal even if their build strings differ.
```rust
# #[macro_use]
# extern crate error_chain;
extern crate semver;
use semver::{Identifier, Version};
#
# error_chain! {
# foreign_links {
# SemVer(semver::SemVerError);
# }
# }
fn run() -> Result<()> {
let version_str = "1.0.49-125+g72ee7853";
let parsed_version = Version::parse(version_str)?;
assert_eq!(
parsed_version,
Version {
major: 1,
minor: 0,
patch: 49,
pre: vec![Identifier::Numeric(125)],
build: vec![],
}
);
assert_eq!(
parsed_version.build,
vec![Identifier::AlphaNumeric(String::from("g72ee7853"))]
);
let serialized_version = parsed_version.to_string();
assert_eq!(&serialized_version, version_str);
Ok(())
}
#
# quick_main!(run);
```
[ex-semver-prerelease]: #ex-semver-prerelease
<a name="ex-semver-prerelease"></a>
## Check if given version is pre-release.
[![semver-badge]][semver] [![cat-config-badge]][cat-config]
Given two versions, we assert (by using [`is_prerelease`]) that one is pre-release and that the other is not.
```rust
# #[macro_use]
# extern crate error_chain;
extern crate semver;
use semver::Version;
#
# error_chain! {
# foreign_links {
# SemVer(semver::SemVerError);
# }
# }
fn run() -> Result<()> {
let version_1 = Version::parse("1.0.0-alpha")?;
let version_2 = Version::parse("1.0.0")?;
assert!(version_1.is_prerelease());
assert!(!version_2.is_prerelease());
Ok(())
}
#
# quick_main!(run);
```
[ex-semver-latest]: #ex-semver-latest
<a name="ex-semver-latest"></a>
## Find the latest version satisfying given range
[![semver-badge]][semver] [![cat-config-badge]][cat-config]
Given a list of version &strs, finds the latest [`semver::Version`] that satisfying a given [`semver::VersionReq`] using [`VersionReq::matches`].
```rust
# #[macro_use]
# extern crate error_chain;
extern crate semver;
use semver::{Version, VersionReq};
#
# error_chain! {
# foreign_links {
# SemVer(semver::SemVerError);
# SemVerReq(semver::ReqParseError);
# }
# }
fn find_max_matching_version<'a, I>(version_req_str: &str, iterable: I) -> Result<Option<Version>>
where
I: IntoIterator<Item = &'a str>,
{
let vreq = VersionReq::parse(version_req_str)?;
Ok(
iterable
.into_iter()
.filter_map(|s| Version::parse(s).ok())
.filter(|s| vreq.matches(s))
.max(),
)
}
fn run() -> Result<()> {
assert_eq!(
find_max_matching_version("<= 1.0.0", vec!["0.9.0", "1.0.0", "1.0.1"])?,
Some(Version::parse("1.0.0")?)
);
// Shows Semver precedence for pre-release tags
assert_eq!(
find_max_matching_version(
">1.2.3-alpha.3",
vec![
"1.2.3-alpha.3",
"1.2.3-alpha.4",
"1.2.3-alpha.10",
"1.2.3-beta.4",
"3.4.5-alpha.9",
]
)?,
Some(Version::parse("1.2.3-beta.4")?)
);
Ok(())
}
#
# quick_main!(run);
```
[ex-semver-command]: #ex-semver-command
<a name="ex-semver-command"></a>
## Check external command version for compatibility
[![semver-badge]][semver] [![cat-text-processing-badge]][cat-text-processing] [![cat-os-badge]][cat-os]
Runs `git --version` using [`Command`], then parses the version number into a [`semver::Version`]
using [`Version::parse`]. A [`semver::VersionReq`] is used to compare the parsed version to a
minimum version requirement.
```rust,no_run
# #[macro_use]
# extern crate error_chain;
extern crate semver;
use std::process::Command;
use semver::{Version, VersionReq};
#
# error_chain! {
# foreign_links {
# Io(std::io::Error);
# Utf8(std::string::FromUtf8Error);
# SemVer(semver::SemVerError);
# SemVerReq(semver::ReqParseError);
# }
# }
fn run() -> Result<()> {
let version_constraint = "> 1.12.0";
let version_test = VersionReq::parse(version_constraint)?;
let output = Command::new("git").arg("--version").output()?;
if !output.status.success() {
bail!("Command executed with failing error code");
}
let stdout = String::from_utf8(output.stdout)?;
// `git --version` output: "git version x.y.z"
let version = stdout.split(" ").last().ok_or_else(|| {
"Invalid command output"
})?;
let parsed_version = Version::parse(version)?;
if !version_test.matches(&parsed_version) {
bail!("Command version lower than minimum supported version (found {}, need {})",
parsed_version, version_constraint);
}
Ok(())
}
#
# quick_main!(run);
```
{{#include links.md}}
<!-- API Reference -->
[`Archive::entries`]: https://docs.rs/tar/*/tar/struct.Archive.html#method.entries
[`Archive::unpack`]: https://docs.rs/tar/*/tar/struct.Archive.html#method.unpack
[`Builder::append_dir_all`]: https://docs.rs/tar/*/tar/struct.Builder.html#method.append_dir_all
[`Command`]: https://doc.rust-lang.org/std/process/struct.Command.html
[`Default`]: https://doc.rust-lang.org/std/default/trait.Default.html
[`Entry::unpack`]: https://docs.rs/tar/*/tar/struct.Entry.html#method.unpack
[`File`]: https://doc.rust-lang.org/std/fs/struct.File.html
[`filter_entry`]: https://docs.rs/walkdir/*/walkdir/struct.IntoIter.html#method.filter_entry
[`follow_links`]: https://docs.rs/walkdir/*/walkdir/struct.WalkDir.html#method.follow_links
[`glob_with`]: https://docs.rs/glob/*/glob/fn.glob_with.html
[`GzDecoder`]: https://docs.rs/flate2/*/flate2/read/struct.GzDecoder.html
[`GzEncoder`]: https://docs.rs/flate2/*/flate2/write/struct.GzEncoder.html
[`is_prerelease`]: https://docs.rs/semver/*/semver/struct.Version.html#method.is_prerelease
[`Iterator::filter`]: https://doc.rust-lang.org/std/iter/trait.Iterator.html#method.filter
[`MatchOptions`]: https://docs.rs/glob/*/glob/struct.MatchOptions.html
[`Path::strip_prefix`]: https://doc.rust-lang.org/std/path/struct.Path.html#method.strip_prefix
[`same_file::Handle`]: https://docs.rs/same-file/*/same_file/struct.Handle.html
[`same_file::is_same_file`]: https://docs.rs/same-file/*/same_file/fn.is_same_file.html#method.is_same_file
[`semver::Version`]: https://docs.rs/semver/*/semver/struct.Version.html
[`semver::VersionReq`]: https://docs.rs/semver/*/semver/struct.VersionReq.html
[`tar::Archive`]: https://docs.rs/tar/*/tar/struct.Archive.html
[`tar::Builder`]: https://docs.rs/tar/*/tar/struct.Builder.html
[`tar::Entries`]: https://docs.rs/tar/*/tar/struct.Entries.html
[`tar::Entry`]: https://docs.rs/tar/*/tar/struct.Entry.html
[`Version::parse`]: https://docs.rs/semver/*/semver/struct.Version.html#method.parse
[`VersionReq::matches`]: https://docs.rs/semver/*/semver/struct.VersionReq.html#method.matches
[`WalkDir::depth`]: https://docs.rs/walkdir/*/walkdir/struct.DirEntry.html#method.depth
[`WalkDir::DirEntry`]: https://docs.rs/walkdir/*/walkdir/struct.DirEntry.html
[`WalkDir::max_depth`]: https://docs.rs/walkdir/*/walkdir/struct.WalkDir.html#method.max_depth
[`WalkDir::min_depth`]: https://docs.rs/walkdir/*/walkdir/struct.WalkDir.html#method.min_depth
<!-- Other Reference -->
[Semantic Versioning Specification]: http://semver.org/

1879
src/basics.md
View file

File diff suppressed because it is too large Load diff

254
src/build_tools.md
View file

@ -1,254 +0,0 @@
# Build Time Tooling
This section covers "build-time" tooling, or code that is run prior to compiling a crate's source code.
Conventionally, build-time code lives in a **build.rs** file and is commonly referred to as a "build script".
Common use cases include rust code generation and compilation of bundled C/C++/asm code.
See crates.io's [documentation on the matter][build-script-docs] for more information.
| Recipe | Crates | Categories |
|--------|--------|------------|
| [Compile and link statically to a bundled C library][ex-cc-static-bundled] | [![cc-badge]][cc] | [![cat-development-tools-badge]][cat-development-tools] |
| [Compile and link statically to a bundled C++ library][ex-cc-static-bundled-cpp] | [![cc-badge]][cc] | [![cat-development-tools-badge]][cat-development-tools] |
| [Compile a C library while setting custom defines][ex-cc-custom-defines] | [![cc-badge]][cc] | [![cat-development-tools-badge]][cat-development-tools] |
[ex-cc-static-bundled]: #ex-cc-static-bundled
<a name="ex-cc-static-bundled"></a>
## Compile and link statically to a bundled C library
[![cc-badge]][cc] [![cat-development-tools-badge]][cat-development-tools]
To accommodate scenarios where additional C, C++, or assembly is required in a project, the [**cc**][cc] crate
offers a simple api for compiling bundled C/C++/asm code into static libraries (**.a**) that can be statically linked to by **rustc**.
The following example has some bundled C code (**src/hello.c**) that will be used from rust.
Before compiling our rust source code, the "build" file (**build.rs**) specified in **Cargo.toml** will run.
Using the [**cc**][cc] crate, a static library file will be produced (in this case, **libhello.a**, see
[`compile` docs][cc-build-compile]) which can then be used from rust by declaring the external function signatures in an `extern` block.
Since the bundled C is very simple, only a single source file needs to be passed to [`cc::Build`][cc-build].
For more complex build requirements, [`cc::Build`][cc-build] offers a full suite of builder methods for specifying
[`include`][cc-build-include] paths and extra compiler [`flag`][cc-build-flag]s.
### `Cargo.toml`
```toml
[package]
...
build = "build.rs"
[build-dependencies]
cc = "1"
[dependencies]
error-chain = "0.11"
```
### `build.rs`
```rust,no_run
extern crate cc;
fn main() {
cc::Build::new()
.file("src/hello.c")
.compile("hello"); // outputs `libhello.a`
}
```
### `src/hello.c`
```c
#include <stdio.h>
void hello() {
printf("Hello from C!\n");
}
void greet(const char* name) {
printf("Hello, %s!\n", name);
}
```
### `src/main.rs`
```rust,ignore
# #[macro_use] extern crate error_chain;
use std::ffi::CString;
use std::os::raw::c_char;
#
# error_chain! {
# foreign_links {
# NulError(::std::ffi::NulError);
# Io(::std::io::Error);
# }
# }
#
# fn prompt(s: &str) -> Result<String> {
# use std::io::Write;
# print!("{}", s);
# std::io::stdout().flush()?;
# let mut input = String::new();
# std::io::stdin().read_line(&mut input)?;
# Ok(input.trim().to_string())
# }
extern {
fn hello();
fn greet(name: *const c_char);
}
fn run() -> Result<()> {
unsafe { hello() }
let name = prompt("What's your name? ")?;
let c_name = CString::new(name)?;
unsafe { greet(c_name.as_ptr()) }
Ok(())
}
#
# quick_main!(run);
```
[ex-cc-static-bundled-cpp]: #ex-cc-static-bundled-cpp
<a name="ex-cc-static-bundled-cpp"></a>
## Compile and link statically to a bundled C++ library
[![cc-badge]][cc] [![cat-development-tools-badge]][cat-development-tools]
Linking a bundled C++ library is very similar to linking a bundled C library. The two core differences when compiling and statically linking a bundled C++ library are specifying a C++ compiler via the builder method [`cpp(true)`][cc-build-cpp] and preventing name mangling by the C++ compiler by adding the `extern "C"` section at the top of our C++ source file.
### `Cargo.toml`
```toml
[package]
...
build = "build.rs"
[build-dependencies]
cc = "1"
```
### `build.rs`
```rust,no_run
extern crate cc;
fn main() {
cc::Build::new()
.cpp(true)
.file("src/foo.cpp")
.compile("foo");
}
```
### `src/foo.cpp`
```cpp
extern "C" {
int multiply(int x, int y);
}
int multiply(int x, int y) {
return x*y;
}
```
### `src/main.rs`
```rust,ignore
extern {
fn multiply(x : i32, y : i32) -> i32;
}
fn main(){
unsafe {
println!("{}", multiply(5,7));
}
}
```
[ex-cc-custom-defines]: #ex-cc-custom-defines
<a name="ex-cc-custom-defines"></a>
## Compile a C library while setting custom defines
[![cc-badge]][cc] [![cat-development-tools-badge]][cat-development-tools]
It is simple to build bundled C code with custom defines using [`cc::Build::define`].
It takes an [`Option`] value, so it is possible to create defines such as `#define APP_NAME "foo"`
as well as `#define WELCOME` (pass `None` as the value for a value-less define). This example builds
a bundled C file with dynamic defines set in `build.rs` and prints "**Welcome to foo - version 1.0.2**"
when run. Cargo sets some [environment variables][cargo-env] which may be useful for some custom defines.
### `Cargo.toml`
```toml
[package]
...
version = "1.0.2"
build = "build.rs"
[build-dependencies]
cc = "1"
```
### `build.rs`
```rust,no_run
extern crate cc;
fn main() {
cc::Build::new()
.define("APP_NAME", "\"foo\"")
.define("VERSION", format!("\"{}\"", env!("CARGO_PKG_VERSION")).as_str())
.define("WELCOME", None)
.file("src/foo.c")
.compile("foo");
}
```
### `src/foo.c`
```c
#include <stdio.h>
void print_app_info() {
#ifdef WELCOME
printf("Welcome to ");
#endif
printf("%s - version %s\n", APP_NAME, VERSION);
}
```
### `src/main.rs`
```rust,ignore
extern {
fn print_app_info();
}
fn main(){
unsafe {
print_app_info();
}
}
```
{{#include links.md}}
<!-- Other Reference -->
[`cc::Build::define`]: https://docs.rs/cc/*/cc/struct.Build.html#method.define
[`Option`]: https://doc.rust-lang.org/std/option/enum.Option.html
[build-script-docs]: http://doc.crates.io/build-script.html
[cargo-env]: http://doc.crates.io/environment-variables.html#environment-variables-cargo-sets-for-crates
[cc-build-compile]: https://docs.rs/cc/*/cc/struct.Build.html#method.compile
[cc-build-cpp]: https://docs.rs/cc/*/cc/struct.Build.html#method.cpp
[cc-build-flag]: https://docs.rs/cc/*/cc/struct.Build.html#method.flag
[cc-build-include]: https://docs.rs/cc/*/cc/struct.Build.html#method.include
[cc-build]: https://docs.rs/cc/*/cc/struct.Build.html
[playground]: https://play.rust-lang.org

8
src/cli.md Normal file
View file

@ -0,0 +1,8 @@
# Command Line
| Recipe | Crates | Categories |
|--------|--------|------------|
| [Parse command line arguments][ex-clap-basic] | [![clap-badge]][clap] | [![cat-command-line-badge]][cat-command-line] |
[ex-clap-basic]: cli/arguments.html#parse-command-line-arguments
{{#include links.md}}

3
src/cli/arguments.md Normal file
View file

@ -0,0 +1,3 @@
{{#include arguments/clap-basic.md}}
{{#include ../links.md}}

85
src/cli/arguments/clap-basic.md Normal file
View file

@ -0,0 +1,85 @@
## Parse command line arguments
[![clap-badge]][clap] [![cat-command-line-badge]][cat-command-line]
This application describes the structure of its command-line interface using
`clap`'s builder style. The [documentation] gives two other possible ways to
instantiate an application.
In the builder style, `with_name` is the unique identifier that `value_of` will
use to retrieve the value passed. The `short` and `long` options control the
flag the user will be expected to type; short flags look like `-f` and long
flags look like `--file`.
```rust
extern crate clap;
use clap::{Arg, App};
fn main() {
let matches = App::new("My Test Program")
.version("0.1.0")
.author("Hackerman Jones <hckrmnjones@hack.gov>")
.about("Teaches argument parsing")
.arg(Arg::with_name("file")
.short("f")
.long("file")
.takes_value(true)
.help("A cool file"))
.arg(Arg::with_name("num")
.short("n")
.long("number")
.takes_value(true)
.help("Five less than your favorite number"))
.get_matches();
let myfile = matches.value_of("file").unwrap_or("input.txt");
println!("The file passed is: {}", myfile);
let num_str = matches.value_of("num");
match num_str {
None => println!("No idea what your favorite number is."),
Some(s) => {
match s.parse::<i32>() {
Ok(n) => println!("Your favorite number must be {}.", n + 5),
Err(_) => println!("That's not a number! {}", s),
}
}
}
}
```
Usage information is generated by `clap`. The usage for the example application
looks like this.
```
My Test Program 0.1.0
Hackerman Jones <hckrmnjones@hack.gov>
Teaches argument parsing
USAGE:
testing [OPTIONS]
FLAGS:
-h, --help Prints help information
-V, --version Prints version information
OPTIONS:
-f, --file <file> A cool file
-n, --number <num> Five less than your favorite number
```
We can test the application by running a command like the following.
```
$ cargo run -- -f myfile.txt -n 251
```
The output is:
```
The file passed is: myfile.txt
Your favorite number must be 256.
```
[documentation]: https://docs.rs/clap/

13
src/compression.md Normal file
View file

@ -0,0 +1,13 @@
# Compression
| Recipe | Crates | Categories |
|--------|--------|------------|
| [Decompress a tarball][ex-tar-decompress] | [![flate2-badge]][flate2] [![tar-badge]][tar] | [![cat-compression-badge]][cat-compression] |
| [Compress a directory into a tarball][ex-tar-compress] | [![flate2-badge]][flate2] [![tar-badge]][tar] | [![cat-compression-badge]][cat-compression] |
| [Decompress a tarball while removing a prefix from the paths][ex-tar-strip-prefix] | [![flate2-badge]][flate2] [![tar-badge]][tar] | [![cat-compression-badge]][cat-compression] |
[ex-tar-decompress]: compression/tar.html#decompress-a-tarball
[ex-tar-compress]: compression/tar.html#compress-a-directory-into-tarball
[ex-tar-strip-prefix]: compression/tar.html#decompress-a-tarball-while-removing-a-prefix-from-the-paths
{{#include links.md}}

9
src/compression/tar.md Normal file
View file

@ -0,0 +1,9 @@
# Working with Tarballs
{{#include tar/tar-decompress.md}}
{{#include tar/tar-compress.md}}
{{#include tar/tar-strip-prefix.md}}
{{#include ../links.md}}

43
src/compression/tar/tar-compress.md Normal file
View file

@ -0,0 +1,43 @@
## Compress a directory into tarball
[![flate2-badge]][flate2] [![tar-badge]][tar] [![cat-compression-badge]][cat-compression]
Compress `/var/log` directory into `archive.tar.gz`.
Creates a [`File`] wrapped in [`GzEncoder`]
and [`tar::Builder`]. </br>Adds contents of `/var/log` directory recursively into the archive
under `backup/logs`path with [`Builder::append_dir_all`].
[`GzEncoder`] is responsible for transparently compressing the
data prior to writing it into `archive.tar.gz`.
```rust,no_run
# #[macro_use]
# extern crate error_chain;
extern crate tar;
extern crate flate2;
#
# error_chain! {
# foreign_links {
# Io(std::io::Error);
# }
# }
use std::fs::File;
use flate2::Compression;
use flate2::write::GzEncoder;
fn run() -> Result<()> {
let tar_gz = File::create("archive.tar.gz")?;
let enc = GzEncoder::new(tar_gz, Compression::default());
let mut tar = tar::Builder::new(enc);
tar.append_dir_all("backup/logs", "/var/log")?;
Ok(())
}
#
# quick_main!(run);
```
[`Builder::append_dir_all`]: https://docs.rs/tar/*/tar/struct.Builder.html#method.append_dir_all
[`File`]: https://doc.rust-lang.org/std/fs/struct.File.html
[`GzEncoder`]: https://docs.rs/flate2/*/flate2/write/struct.GzEncoder.html
[`tar::Builder`]: https://docs.rs/tar/*/tar/struct.Builder.html

41
src/compression/tar/tar-decompress.md Normal file
View file

@ -0,0 +1,41 @@
## Decompress a tarball
[![flate2-badge]][flate2] [![tar-badge]][tar] [![cat-compression-badge]][cat-compression]
Decompress ([`GzDecoder`]) and
extract ([`Archive::unpack`]) all files from a compressed tarball
named `archive.tar.gz` located in the current working directory
to the same location.
```rust,no_run
# #[macro_use]
# extern crate error_chain;
extern crate flate2;
extern crate tar;
use std::fs::File;
use flate2::read::GzDecoder;
use tar::Archive;
#
# error_chain! {
# foreign_links {
# Io(std::io::Error);
# }
# }
fn run() -> Result<()> {
let path = "archive.tar.gz";
let tar_gz = File::open(path)?;
let tar = GzDecoder::new(tar_gz);
let mut archive = Archive::new(tar);
archive.unpack(".")?;
Ok(())
}
#
# quick_main!(run);
```
[`Archive::unpack`]: https://docs.rs/tar/*/tar/struct.Archive.html#method.unpack
[`GzDecoder`]: https://docs.rs/flate2/*/flate2/read/struct.GzDecoder.html

53
src/compression/tar/tar-strip-prefix.md Normal file
View file

@ -0,0 +1,53 @@
## Decompress a tarball while removing a prefix from the paths
[![flate2-badge]][flate2] [![tar-badge]][tar] [![cat-compression-badge]][cat-compression]
Iterate over the [`Archive::entries`]. Use [`Path::strip_prefix`] to remove
the specified path prefix (`bundle/logs`). Finally, extract the [`tar::Entry`]
via [`Entry::unpack`].
```rust,no_run
# #[macro_use]
# extern crate error_chain;
extern crate flate2;
extern crate tar;
#
# error_chain! {
# foreign_links {
# Io(std::io::Error);
# StripPrefixError(::std::path::StripPrefixError);
# }
# }
use std::fs::File;
use std::path::PathBuf;
use flate2::read::GzDecoder;
use tar::Archive;
fn run() -> Result<()> {
let file = File::open("archive.tar.gz")?;
let mut archive = Archive::new(GzDecoder::new(file));
let prefix = "bundle/logs";
println!("Extracted the following files:");
archive
.entries()?
.filter_map(|e| e.ok())
.map(|mut entry| -> Result<PathBuf> {
let path = entry.path()?.strip_prefix(prefix)?.to_owned();
entry.unpack(&path)?;
Ok(path)
})
.filter_map(|e| e.ok())
.for_each(|x| println!("> {}", x.display()));
Ok(())
}
#
# quick_main!(run);
```
[`Archive::entries`]: https://docs.rs/tar/*/tar/struct.Archive.html#method.entries
[`Entry::unpack`]: https://docs.rs/tar/*/tar/struct.Entry.html#method.unpack
[`Path::strip_prefix`]: https://doc.rust-lang.org/std/path/struct.Path.html#method.strip_prefix
[`tar::Entry`]: https://docs.rs/tar/*/tar/struct.Entry.html

564
src/concurrency.md
View file

@ -2,563 +2,27 @@
| Recipe | Crates | Categories |
|--------|--------|------------|
| [Spawn a short-lived thread][ex-crossbeam-spawn] | [![crossbeam-badge]][crossbeam] | [![cat-concurrency-badge]][cat-concurrency] |
| [Maintain global mutable state][ex-global-mut-state] | [![lazy_static-badge]][lazy_static] | [![cat-rust-patterns-badge]][cat-rust-patterns] |
| [Calculate SHA1 sum of *.iso files concurrently][ex-threadpool-walk] | [![threadpool-badge]][threadpool] [![walkdir-badge]][walkdir] [![num_cpus-badge]][num_cpus] [![ring-badge]][ring] | [![cat-concurrency-badge]][cat-concurrency][![cat-filesystem-badge]][cat-filesystem] |
| [Draw fractal dispatching work to a thread pool][ex-threadpool-fractal] | [![threadpool-badge]][threadpool] [![num-badge]][num] [![num_cpus-badge]][num_cpus] [![image-badge]][image] | [![cat-concurrency-badge]][cat-concurrency][![cat-science-badge]][cat-science][![cat-rendering-badge]][cat-rendering] |
| [Mutate the elements of an array in parallel][ex-rayon-iter-mut] | [![rayon-badge]][rayon] | [![cat-concurrency-badge]][cat-concurrency] |
| [Test in parallel if any or all elements of a collection match a given predicate][ex-rayon-any-all] | [![rayon-badge]][rayon] | [![cat-concurrency-badge]][cat-concurrency] |
| [Search items using given predicate in parallel][ex-rayon-parallel-search] | [![rayon-badge]][rayon] | [![cat-concurrency-badge]][cat-concurrency] |
| [Sort a vector in parallel][ex-rayon-parallel-sort] | [![rayon-badge]][rayon] [![rand-badge]][rand] | [![cat-concurrency-badge]][cat-concurrency] |
| [Map-reduce in parallel][ex-rayon-map-reduce] | [![rayon-badge]][rayon] | [![cat-concurrency-badge]][cat-concurrency] |
| [Generate jpg thumbnails in parallel][ex-rayon-thumbnails] | [![rayon-badge]][rayon] [![glob-badge]][glob] [![image-badge]][image] | [![cat-concurrency-badge]][cat-concurrency][![cat-filesystem-badge]][cat-filesystem] |
| [Spawn a short-lived thread][ex-crossbeam-spawn] | [![crossbeam-badge]][crossbeam] | [![cat-concurrency-badge]][cat-concurrency] |
| [Draw fractal dispatching work to a thread pool][ex-threadpool-fractal] | [![threadpool-badge]][threadpool] [![num-badge]][num] [![num_cpus-badge]][num_cpus] [![image-badge]][image] | [![cat-concurrency-badge]][cat-concurrency][![cat-science-badge]][cat-science][![cat-rendering-badge]][cat-rendering] |
| [Calculate SHA1 sum of *.iso files concurrently][ex-threadpool-walk] | [![threadpool-badge]][threadpool] [![walkdir-badge]][walkdir] [![num_cpus-badge]][num_cpus] [![ring-badge]][ring] | [![cat-concurrency-badge]][cat-concurrency][![cat-filesystem-badge]][cat-filesystem] |
[ex-rayon-iter-mut]: #ex-rayon-iter-mut
<a name="ex-rayon-iter-mut"></a>
## Mutate the elements of an array in parallel
[![rayon-badge]][rayon] [![cat-concurrency-badge]][cat-concurrency]
The example uses the `rayon` crate, which is a data parallelism library for Rust.
`rayon` provides the [`par_iter_mut`] method for any parallel iterable data type.
It lets us write iterator-like chains that execute in parallel.
```rust
extern crate rayon;
use rayon::prelude::*;
fn main() {
let mut arr = [0, 7, 9, 11];
arr.par_iter_mut().for_each(|p| *p -= 1);
println!("{:?}", arr);
}
```
[ex-rayon-any-all]: #ex-rayon-any-all
<a name="ex-rayon-any-all"></a>
## Test in parallel if any or all elements of a collection match a given predicate
[![rayon-badge]][rayon] [![cat-concurrency-badge]][cat-concurrency]
This example demonstrates using the [`rayon::any`] and [`rayon::all`] methods, which are parallelized counterparts to [`std::any`] and [`std::all`]. [`rayon::any`] checks in parallel whether any element of the iterator matches the predicate, and returns as soon as one is found. [`rayon::all`] checks in parallel whether all elements of the iterator match the predicate, and returns as soon as a non-matching element is found.
```rust
extern crate rayon;
use rayon::prelude::*;
fn main() {
let mut vec = vec![2, 4, 6, 8];
assert!(!vec.par_iter().any(|n| (*n % 2) != 0)); // None are odd.
assert!(vec.par_iter().all(|n| (*n % 2) == 0)); // All are even.
assert!(!vec.par_iter().any(|n| *n > 8 )); // None are greater than 8.
assert!(vec.par_iter().all(|n| *n <= 8 )); // All are less than or equal to 8.
vec.push(9);
assert!(vec.par_iter().any(|n| (*n % 2) != 0)); // At least 1 is odd.
assert!(!vec.par_iter().all(|n| (*n % 2) == 0)); // Not all are even.
assert!(vec.par_iter().any(|n| *n > 8 )); // At least 1 is greater than 8.
assert!(!vec.par_iter().all(|n| *n <= 8 )); // Not all are less than or equal to 8.
}
```
[ex-rayon-parallel-sort]: #ex-rayon-parallel-sort
<a name="ex-rayon-parallel-sort"></a>
## Sort a vector in parallel
[![rayon-badge]][rayon] [![rand-badge]][rand] [![cat-concurrency-badge]][cat-concurrency]
This example will sort in parallel a vector of Strings.
[1] We start by preallocating a vector of empty Strings, so we can mutate the information in parallel later,
to populate the vector with random Strings.
[2] `par_iter_mut().for_each` takes a closure and applies it in parallel on all the elements of the vector.<br/>
[3] Inside the passed closure we modify the element in the vector with a 5 character-long String, random generated.
[4] We have [`multiple options`] to sort an Iterable data type, we chose here to use [`par_sort_unstable`]
because it is usually faster than [`stable sorting`] algorithms which `rayon` also supports.
```rust
extern crate rand;
extern crate rayon;
use rand::{Rng, thread_rng};
use rand::distributions::Alphanumeric;
use rayon::prelude::*;
fn main() {
// [1]
let mut vec = vec![String::new(); 100_000];
// [2]
vec.par_iter_mut().for_each(|p| {
// [3]
let mut rng = thread_rng();
*p = (0..5).map(|_| rng.sample(&Alphanumeric)).collect()
});
// [4]
vec.par_sort_unstable();
}
```
[ex-rayon-map-reduce]: #ex-rayon-map-reduce
<a name="ex-rayon-map-reduce"></a>
## Map-reduce in parallel
[![rayon-badge]][rayon] [![cat-concurrency-badge]][cat-concurrency]
This example uses [`rayon::filter`], [`rayon::map`], and [`rayon::reduce`]
to calculate the conditional average age of a vector of `Person` objects.
[`rayon::filter`] allows us to (in parallel) conditionally include elements from
a collection that satisfy the given predicate. Similarly, [`rayon::map`] and
[`rayon::reduce`] allow us to transform the filtered elements via a unary
operation and reduce them to a single value via a given binary operation,
respectively. We also show use of [`rayon::sum`], which has the same result as
the reduce operation in this example.
```rust
extern crate rayon;
use rayon::prelude::*;
struct Person {
age: u32,
}
fn main() {
let v: Vec<Person> = vec![
Person { age: 23 },
Person { age: 19 },
Person { age: 42 },
Person { age: 17 },
Person { age: 17 },
Person { age: 31 },
Person { age: 30 },
];
let num_over_30 = v.par_iter().filter(|&x| x.age > 30).count() as f32;
let sum_over_30 = v.par_iter()
.map(|x| x.age)
.filter(|&x| x > 30)
.reduce(|| 0, |x, y| x + y);
let alt_sum_30: u32 = v.par_iter()
.map(|x| x.age)
.filter(|&x| x > 30)
.sum();
let avg_over_30 = sum_over_30 as f32 / num_over_30;
let alt_avg_over_30 = alt_sum_30 as f32/ num_over_30;
assert!((avg_over_30 - alt_avg_over_30).abs() < std::f32::EPSILON);
println!("The average age of people older than 30 is {}", avg_over_30);
}
```
[ex-rayon-parallel-search]: #ex-rayon-parallel-search
<a name="ex-rayon-parallel-search"></a>
## Search items using given predicate in parallel
[![rayon-badge]][rayon] [![cat-concurrency-badge]][cat-concurrency]
This example uses [`rayon::find_any`] and [`par_iter`] to search a vector in
parallel for an element satisfying the predicate in the given closure.
If there are multiple elements satisfying the predicate defined in the closure
argument of [`rayon::find_any`], we are only guaranteed that one of them will be
found, but not necessarily that the first will be found.
Also note that the argument to the closure is a reference to a reference
(`&&x`). Please see the discussion on [`std::find`] for additional details.
```rust
extern crate rayon;
use rayon::prelude::*;
fn main() {
let v = vec![6, 2, 1, 9, 3, 8, 11];
let f1 = v.par_iter().find_any(|&&x| x == 9);
let f2 = v.par_iter().find_any(|&&x| x % 2 == 0 && x > 6);
let f3 = v.par_iter().find_any(|&&x| x > 8);
assert_eq!(f1, Some(&9));
assert_eq!(f2, Some(&8));
assert!(f3 > Some(&8));
}
```
[ex-rayon-thumbnails]: #ex-rayon-thumbnails
<a name="ex-rayon-thumbnails"></a>
## Generate jpg thumbnails in parallel
[![rayon-badge]][rayon] [![glob-badge]][glob] [![image-badge]][image] [![cat-concurrency-badge]][cat-concurrency] [![cat-filesystem-badge]][cat-filesystem]
This example generates thumbnails for all .jpg in the current directory and saves them in a new folder called `thumbnails`.
Files are found using [`glob::glob_with`] to match case insensitively on both `.jpg` and `.JPG`. `rayon` is then used to resize images in parallel using [`par_iter`] along with the `make_thumbnail()` helper function which internally uses [`DynamicImage::resize`].
```rust,no_run
# #[macro_use]
# extern crate error_chain;
extern crate glob;
extern crate image;
extern crate rayon;
use std::path::Path;
use std::fs::create_dir_all;
# use error_chain::ChainedError;
use glob::{glob_with, MatchOptions};
use image::{FilterType, ImageError};
use rayon::prelude::*;
# error_chain! {
# foreign_links {
# Image(ImageError);
# Io(std::io::Error);
# Glob(glob::PatternError);
# }
# }
fn run() -> Result<()> {
// find all files in current directory that have a .jpg extension
// use the default MatchOptions so the search is case insensitive
let options: MatchOptions = Default::default();
let files: Vec<_> = glob_with("*.jpg", &options)?
.filter_map(|x| x.ok())
.collect();
if files.len() == 0 {
bail!("No .jpg files found in current directory");
}
let thumb_dir = "thumbnails";
create_dir_all(thumb_dir)?;
println!("Saving {} thumbnails into '{}'...", files.len(), thumb_dir);
let image_failures: Vec<_> = files
.par_iter()
.map(|path| {
make_thumbnail(path, thumb_dir, 300)
.map_err(|e| e.chain_err(|| path.display().to_string()))
})
.filter_map(|x| x.err())
.collect();
image_failures.iter().for_each(|x| println!("{}", x.display_chain()));
println!("{} thumbnails saved successfully", files.len() - image_failures.len());
Ok(())
}
/// Resize `original` to have a maximum dimension of `longest_edge` and save the
/// resized image to the `thumb_dir` folder
fn make_thumbnail<PA, PB>(original: PA, thumb_dir: PB, longest_edge: u32) -> Result<()>
where
PA: AsRef<Path>,
PB: AsRef<Path>,
{
let img = image::open(original.as_ref())?;
let file_path = thumb_dir.as_ref().join(original);
Ok(img.resize(longest_edge, longest_edge, FilterType::Nearest)
.save(file_path)?)
}
#
# quick_main!(run);
```
[ex-crossbeam-spawn]: #ex-crossbeam-spawn
<a name="ex-crossbeam-spawn"></a>
## Spawn a short-lived thread
[![crossbeam-badge]][crossbeam] [![cat-concurrency-badge]][cat-concurrency]
The example uses the `crossbeam` crate, which provides data structures and functions
for concurrent and parallel programming. [`Scope::spawn`] spawns a new scoped thread that is guaranteed
to terminate before returning from the closure that passed into [`crossbeam::scope`] function, meaning that
you can reference data from the calling function.
```rust
extern crate crossbeam;
use std::cmp;
fn main() {
let arr = &[-4, 1, 10, 25];
let max = find_max(arr, 0, arr.len());
assert_eq!(25, max);
}
fn find_max(arr: &[i32], start: usize, end: usize) -> i32 {
// Perform sequential computation if there are only a few elements.
const THRESHOLD: usize = 2;
if end - start <= THRESHOLD {
return *arr.iter().max().unwrap();
}
let mid = start + (end - start) / 2;
crossbeam::scope(|scope| {
let left = scope.spawn(|| find_max(arr, start, mid));
let right = scope.spawn(|| find_max(arr, mid, end));
cmp::max(left.join(), right.join())
})
}
```
[ex-threadpool-fractal]: #ex-threadpool-fractal
<a name="ex-threadpool-fractal"></a>
## Draw fractal dispatching work to a thread pool
[![threadpool-badge]][threadpool] [![num-badge]][num] [![num_cpus-badge]][num_cpus] [![image-badge]][image] [![cat-concurrency-badge]][cat-concurrency][![cat-science-badge]][cat-science][![cat-rendering-badge]][cat-rendering]
This example draws a fractal from [Julia set] to an image utilizing a thread pool for computation.
<a href="https://cloud.githubusercontent.com/assets/221000/26546700/9be34e80-446b-11e7-81dc-dd9871614ea1.png"><img src="https://cloud.githubusercontent.com/assets/221000/26546700/9be34e80-446b-11e7-81dc-dd9871614ea1.png" width="150" /></a>
Firstly, the example allocates memory for output image of given width and height with [`ImageBuffer::new`]
and pre-calculates all possible RGB pixel values using [`Rgb::from_channels`].
Secondly, creates a new [`ThreadPool`] with thread count equal to number of
logical cores in CPU obtained with [`num_cpus::get`].
Subsequently, dispatches calculation to thread pool [`ThreadPool::execute`].
Lastly, collects calculation results via [`mpsc::channel`] with [`Receiver::recv`], draws them with [`ImageBuffer::put_pixel`] and encodes the final image into `output.png` using [`ImageBuffer::save`].
```rust,no_run
# #[macro_use]
# extern crate error_chain;
extern crate threadpool;
extern crate num;
extern crate num_cpus;
extern crate image;
use std::sync::mpsc::{channel, RecvError};
use threadpool::ThreadPool;
use num::complex::Complex;
use image::{ImageBuffer, Pixel, Rgb};
#
# error_chain! {
# foreign_links {
# MpscRecv(RecvError);
# Io(std::io::Error);
# }
# }
#
# // Function converting intensity values to RGB
# // Based on http://www.efg2.com/Lab/ScienceAndEngineering/Spectra.htm
# fn wavelength_to_rgb(wavelength: u32) -> Rgb<u8> {
# let wave = wavelength as f32;
#
# let (r, g, b) = match wavelength {
# 380...439 => ((440. - wave) / (440. - 380.), 0.0, 1.0),
# 440...489 => (0.0, (wave - 440.) / (490. - 440.), 1.0),
# 490...509 => (0.0, 1.0, (510. - wave) / (510. - 490.)),
# 510...579 => ((wave - 510.) / (580. - 510.), 1.0, 0.0),
# 580...644 => (1.0, (645. - wave) / (645. - 580.), 0.0),
# 645...780 => (1.0, 0.0, 0.0),
# _ => (0.0, 0.0, 0.0),
# };
#
# let factor = match wavelength {
# 380...419 => 0.3 + 0.7 * (wave - 380.) / (420. - 380.),
# 701...780 => 0.3 + 0.7 * (780. - wave) / (780. - 700.),
# _ => 1.0,
# };
#
# let (r, g, b) = (normalize(r, factor), normalize(g, factor), normalize(b, factor));
# Rgb::from_channels(r, g, b, 0)
# }
#
# // Maps Julia set distance estimation to intensity values
# fn julia(c: Complex<f32>, x: u32, y: u32, width: u32, height: u32, max_iter: u32) -> u32 {
# let width = width as f32;
# let height = height as f32;
#
# let mut z = Complex {
# // scale and translate the point to image coordinates
# re: 3.0 * (x as f32 - 0.5 * width) / width,
# im: 2.0 * (y as f32 - 0.5 * height) / height,
# };
#
# let mut i = 0;
# for t in 0..max_iter {
# if z.norm() >= 2.0 {
# break;
# }
# z = z * z + c;
# i = t;
# }
# i
# }
#
# // Normalizes color intensity values within RGB range
# fn normalize(color: f32, factor: f32) -> u8 {
# ((color * factor).powf(0.8) * 255.) as u8
# }
fn run() -> Result<()> {
let (width, height) = (1920, 1080);
let mut img = ImageBuffer::new(width, height);
let iterations = 300;
// constants to tweak for appearance
let c = Complex::new(-0.8, 0.156);
let pool = ThreadPool::new(num_cpus::get());
let (tx, rx) = channel();
for y in 0..height {
let tx = tx.clone();
pool.execute(move || for x in 0..width {
let i = julia(c, x, y, width, height, iterations);
let pixel = wavelength_to_rgb(380 + i * 400 / iterations);
tx.send((x, y, pixel)).expect("Could not send data!");
});
}
for _ in 0..(width * height) {
let (x, y, pixel) = rx.recv()?;
img.put_pixel(x, y, pixel);
}
let _ = img.save("output.png")?;
Ok(())
}
#
# quick_main!(run);
```
[ex-threadpool-walk]: #ex-threadpool-walk
<a name="ex-threadpool-walk"></a>
## Calculate SHA1 sum of *.iso files concurrently
[![threadpool-badge]][threadpool] [![num_cpus-badge]][num_cpus] [![walkdir-badge]][walkdir] [![ring-badge]][ring] [![cat-concurrency-badge]][cat-concurrency][![cat-filesystem-badge]][cat-filesystem]
This example calculates the SHA1 for every file present in the current directory. A threadpool is created using the number of cpus present in the system with [`num_cpus::get`]. Then every returned by [`Walkdir::new`] is passed into this pool to perform the operations of reading and computing SHA1. At the end the program waits for all jobs to finish. To get better results, compile this program in release mode.
```rust,no_run
# #[macro_use]
# extern crate error_chain;
extern crate walkdir;
extern crate ring;
extern crate num_cpus;
extern crate threadpool;
# error_chain! {
# foreign_links {
# Io(std::io::Error);
# }
# }
#
use walkdir::WalkDir;
use std::fs::File;
use std::io::{BufReader, Read};
use std::path::Path;
use threadpool::ThreadPool;
use std::sync::mpsc::channel;
use ring::digest::{Context, Digest, SHA1};
# // Verify the iso extension
# fn is_iso(entry: &Path) -> bool {
# match entry.extension() {
# Some(e) if e.to_string_lossy().to_lowercase() == "iso" => true,
# _ => false,
# }
# }
#
fn compute_digest<P: AsRef<Path>>(filepath: P) -> Result<(Digest, P)> {
let mut buf_reader = BufReader::new(File::open(&filepath)?);
let mut context = Context::new(&SHA1);
let mut buffer = [0; 1024];
loop {
let count = buf_reader.read(&mut buffer)?;
if count == 0 {
break;
}
context.update(&buffer[..count]);
}
Ok((context.finish(), filepath))
}
fn run() -> Result<()> {
let pool = ThreadPool::new(num_cpus::get());
let (tx, rx) = channel();
// Look in the current directory.
for entry in WalkDir::new("/home/user/Downloads")
.follow_links(true)
.into_iter()
.filter_map(|e| e.ok())
.filter(|e| !e.path().is_dir() && is_iso(e.path())) {
let path = entry.path().to_owned();
let tx = tx.clone();
pool.execute(move || {
let digest = compute_digest(path);
tx.send(digest).expect("Could not send data!");
});
}
drop(tx);
for t in rx.iter() {
let (sha, path) = t?;
println!("{:?} {:?}", sha, path);
}
Ok(())
}
#
# quick_main!(run);
```
[ex-crossbeam-spawn]: concurrency/threads.html#spawn-a-short-lived-thread
[ex-global-mut-state]: concurrency/threads.html#maintain-global-mutable-state
[ex-threadpool-walk]: concurrency/threads.html#calculate-sha1-sum-of-iso-files-concurrently
[ex-threadpool-fractal]: concurrency/threads.html#draw-fractal-dispatching-work-to-a-thread-pool
[ex-rayon-iter-mut]: concurrency/parallel.html#mutate-the-elements-of-an-array-in-parallel
[ex-rayon-any-all]: concurrency/parallel.html#test-in-parallel-if-any-or-all-elements-of-a-collection-match-a-given-predicate
[ex-rayon-parallel-search]: concurrency/parallel.html#search-items-using-given-predicate-in-parallel
[ex-rayon-parallel-sort]: concurrency/parallel.html#sort-a-vector-in-parallel
[ex-rayon-map-reduce]: concurrency/parallel.html#map-reduce-in-parallel
[ex-rayon-thumbnails]: concurrency/parallel.html#generate-jpg-thumbnails-in-parallel
{{#include links.md}}
<!-- API Reference -->
[`crossbeam::scope`]: https://docs.rs/crossbeam/*/crossbeam/fn.scope.html
[`DynamicImage::resize`]: https://docs.rs/image/*/image/enum.DynamicImage.html#method.resize
[`glob::glob_with`]: https://docs.rs/glob/*/glob/fn.glob_with.html
[`ImageBuffer::new`]: https://docs.rs/image/*/image/struct.ImageBuffer.html#method.new
[`ImageBuffer::put_pixel`]: https://docs.rs/image/*/image/struct.ImageBuffer.html#method.put_pixel
[`ImageBuffer::save`]: https://docs.rs/image/*/image/struct.ImageBuffer.html#method.save
[`Iterator`]: https://doc.rust-lang.org/std/iter/trait.Iterator.html
[`mpsc::channel`]: https://doc.rust-lang.org/std/sync/mpsc/fn.channel.html
[`multiple options`]: https://docs.rs/rayon/*/rayon/slice/trait.ParallelSliceMut.html
[`num_cpus::get`]: https://docs.rs/num_cpus/*/num_cpus/fn.get.html
[`par_iter_mut`]: https://docs.rs/rayon/*/rayon/iter/trait.IntoParallelRefMutIterator.html#tymethod.par_iter_mut
[`par_iter`]: https://docs.rs/rayon/*/rayon/iter/trait.IntoParallelRefIterator.html#tymethod.par_iter
[`par_sort_unstable`]: https://docs.rs/rayon/*/rayon/slice/trait.ParallelSliceMut.html#method.par_sort_unstable
[`ParallelIterator`]: https://docs.rs/rayon/*/rayon/iter/trait.ParallelIterator.html
[`rayon::all`]: https://docs.rs/rayon/*/rayon/iter/trait.ParallelIterator.html#method.all
[`rayon::any`]: https://docs.rs/rayon/*/rayon/iter/trait.ParallelIterator.html#method.any
[`rayon::filter`]: https://docs.rs/rayon/*/rayon/iter/trait.ParallelIterator.html#method.filter
[`rayon::find_any`]: https://docs.rs/rayon/*/rayon/iter/trait.ParallelIterator.html#method.find_any
[`rayon::map`]: https://docs.rs/rayon/*/rayon/iter/trait.ParallelIterator.html#method.map
[`rayon::reduce`]: https://docs.rs/rayon/*/rayon/iter/trait.ParallelIterator.html#method.reduce
[`rayon::sum`]: https://docs.rs/rayon/*/rayon/iter/trait.ParallelIterator.html#method.sum
[`Receiver::recv`]: https://doc.rust-lang.org/std/sync/mpsc/struct.Receiver.html#method.recv
[`Rgb::from_channels`]: https://docs.rs/image/*/image/struct.Rgb.html#method.from_channels
[`Scope::spawn`]: https://docs.rs/crossbeam/*/crossbeam/struct.Scope.html#method.spawn
[`stable sorting`]: https://docs.rs/rayon/*/rayon/slice/trait.ParallelSliceMut.html#method.par_sort
[`std::all`]: https://doc.rust-lang.org/std/iter/trait.Iterator.html#method.all
[`std::any`]: https://doc.rust-lang.org/std/iter/trait.Iterator.html#method.any
[`std::find`]: https://doc.rust-lang.org/std/iter/trait.Iterator.html#method.find
[`ThreadPool::execute`]: https://docs.rs/threadpool/*/threadpool/struct.ThreadPool.html#method.execute
[`ThreadPool`]: https://docs.rs/threadpool/*/threadpool/struct.ThreadPool.html
[`Walkdir::new`]: https://docs.rs/walkdir/*/walkdir/struct.WalkDir.html#method.new
<!-- Other Reference -->
[Julia set]: https://en.wikipedia.org/wiki/Julia_set

15
src/concurrency/parallel.md Normal file
View file

@ -0,0 +1,15 @@
# Parallel Tasks
{{#include parallel/rayon-iter-mut.md}}
{{#include parallel/rayon-any-all.md}}
{{#include parallel/rayon-parallel-search.md}}
{{#include parallel/rayon-parallel-sort.md}}
{{#include parallel/rayon-map-reduce.md}}
{{#include parallel/rayon-thumbnails.md}}
{{#include ../links.md}}

32
src/concurrency/parallel/rayon-any-all.md Normal file
View file

@ -0,0 +1,32 @@
## Test in parallel if any or all elements of a collection match a given predicate
[![rayon-badge]][rayon] [![cat-concurrency-badge]][cat-concurrency]
This example demonstrates using the [`rayon::any`] and [`rayon::all`] methods, which are parallelized counterparts to [`std::any`] and [`std::all`]. [`rayon::any`] checks in parallel whether any element of the iterator matches the predicate, and returns as soon as one is found. [`rayon::all`] checks in parallel whether all elements of the iterator match the predicate, and returns as soon as a non-matching element is found.
```rust
extern crate rayon;
use rayon::prelude::*;
fn main() {
let mut vec = vec![2, 4, 6, 8];
assert!(!vec.par_iter().any(|n| (*n % 2) != 0));
assert!(vec.par_iter().all(|n| (*n % 2) == 0));
assert!(!vec.par_iter().any(|n| *n > 8 ));
assert!(vec.par_iter().all(|n| *n <= 8 ));
vec.push(9);
assert!(vec.par_iter().any(|n| (*n % 2) != 0));
assert!(!vec.par_iter().all(|n| (*n % 2) == 0));
assert!(vec.par_iter().any(|n| *n > 8 ));
assert!(!vec.par_iter().all(|n| *n <= 8 ));
}
```
[`rayon::all`]: https://docs.rs/rayon/*/rayon/iter/trait.ParallelIterator.html#method.all
[`rayon::any`]: https://docs.rs/rayon/*/rayon/iter/trait.ParallelIterator.html#method.any
[`std::all`]: https://doc.rust-lang.org/std/iter/trait.Iterator.html#method.all
[`std::any`]: https://doc.rust-lang.org/std/iter/trait.Iterator.html#method.any

21
src/concurrency/parallel/rayon-iter-mut.md Normal file
View file

@ -0,0 +1,21 @@
## Mutate the elements of an array in parallel
[![rayon-badge]][rayon] [![cat-concurrency-badge]][cat-concurrency]
The example uses the `rayon` crate, which is a data parallelism library for Rust.
`rayon` provides the [`par_iter_mut`] method for any parallel iterable data type.
This is an iterator-like chain that potentially executes in parallel.
```rust
extern crate rayon;
use rayon::prelude::*;
fn main() {
let mut arr = [0, 7, 9, 11];
arr.par_iter_mut().for_each(|p| *p -= 1);
println!("{:?}", arr);
}
```
[`par_iter_mut`]: https://docs.rs/rayon/*/rayon/iter/trait.IntoParallelRefMutIterator.html#tymethod.par_iter_mut

56
src/concurrency/parallel/rayon-map-reduce.md Normal file
View file

@ -0,0 +1,56 @@
## Map-reduce in parallel
[![rayon-badge]][rayon] [![cat-concurrency-badge]][cat-concurrency]
This example uses [`rayon::filter`], [`rayon::map`], and [`rayon::reduce`]
to calculate the average age of `Person` objects whose age is over 30.
[`rayon::filter`] returns elements from a collection that satisfy the given
predicate. [`rayon::map`] performs an operation on every element, creating a
new iteration, and [`rayon::reduce`] performs an operation given the previous
reduction and the current element. Also shows use of [`rayon::sum`],
which has the same result as the reduce operation in this example.
```rust
extern crate rayon;
use rayon::prelude::*;
struct Person {
age: u32,
}
fn main() {
let v: Vec<Person> = vec![
Person { age: 23 },
Person { age: 19 },
Person { age: 42 },
Person { age: 17 },
Person { age: 17 },
Person { age: 31 },
Person { age: 30 },
];
let num_over_30 = v.par_iter().filter(|&x| x.age > 30).count() as f32;
let sum_over_30 = v.par_iter()
.map(|x| x.age)
.filter(|&x| x > 30)
.reduce(|| 0, |x, y| x + y);
let alt_sum_30: u32 = v.par_iter()
.map(|x| x.age)
.filter(|&x| x > 30)
.sum();
let avg_over_30 = sum_over_30 as f32 / num_over_30;
let alt_avg_over_30 = alt_sum_30 as f32/ num_over_30;
assert!((avg_over_30 - alt_avg_over_30).abs() < std::f32::EPSILON);
println!("The average age of people older than 30 is {}", avg_over_30);
}
```
[`rayon::filter`]: https://docs.rs/rayon/*/rayon/iter/trait.ParallelIterator.html#method.filter
[`rayon::map`]: https://docs.rs/rayon/*/rayon/iter/trait.ParallelIterator.html#method.map
[`rayon::reduce`]: https://docs.rs/rayon/*/rayon/iter/trait.ParallelIterator.html#method.reduce
[`rayon::sum`]: https://docs.rs/rayon/*/rayon/iter/trait.ParallelIterator.html#method.sum

35
src/concurrency/parallel/rayon-parallel-search.md Normal file
View file

@ -0,0 +1,35 @@
## Search items using given predicate in parallel
[![rayon-badge]][rayon] [![cat-concurrency-badge]][cat-concurrency]
This example uses [`rayon::find_any`] and [`par_iter`] to search a vector in
parallel for an element satisfying the predicate in the given closure.
If there are multiple elements satisfying the predicate defined in the closure
argument of [`rayon::find_any`], `rayon` returns the first one found, not
necessarily the first one.
Also note that the argument to the closure is a reference to a reference
(`&&x`). See the discussion on [`std::find`] for additional details.
```rust
extern crate rayon;
use rayon::prelude::*;
fn main() {
let v = vec![6, 2, 1, 9, 3, 8, 11];
let f1 = v.par_iter().find_any(|&&x| x == 9);
let f2 = v.par_iter().find_any(|&&x| x % 2 == 0 && x > 6);
let f3 = v.par_iter().find_any(|&&x| x > 8);
assert_eq!(f1, Some(&9));
assert_eq!(f2, Some(&8));
assert!(f3 > Some(&8));
}
```
[`par_iter`]: https://docs.rs/rayon/*/rayon/iter/trait.IntoParallelRefIterator.html#tymethod.par_iter
[`rayon::find_any`]: https://docs.rs/rayon/*/rayon/iter/trait.ParallelIterator.html#method.find_any
[`std::find`]: https://doc.rust-lang.org/std/iter/trait.Iterator.html#method.find

32
src/concurrency/parallel/rayon-parallel-sort.md Normal file
View file

@ -0,0 +1,32 @@
## Sort a vector in parallel
[![rayon-badge]][rayon] [![rand-badge]][rand] [![cat-concurrency-badge]][cat-concurrency]
This example will sort in parallel a vector of Strings.
Allocate a vector of empty Strings. `par_iter_mut().for_each` populates random
values in parallel. Although [multiple options]
exist to sort an enumerable data type, [`par_sort_unstable`]
is usually faster than [stable sorting] algorithms.
```rust
extern crate rand;
extern crate rayon;
use rand::{Rng, thread_rng};
use rand::distributions::Alphanumeric;
use rayon::prelude::*;
fn main() {
let mut vec = vec![String::new(); 100_000];
vec.par_iter_mut().for_each(|p| {
let mut rng = thread_rng();
*p = (0..5).map(|_| rng.sample(&Alphanumeric)).collect()
});
vec.par_sort_unstable();
}
```
[`par_sort_unstable`]: https://docs.rs/rayon/*/rayon/slice/trait.ParallelSliceMut.html#method.par_sort_unstable
[multiple options]: https://docs.rs/rayon/*/rayon/slice/trait.ParallelSliceMut.html
[stable sorting]: https://docs.rs/rayon/*/rayon/slice/trait.ParallelSliceMut.html#method.par_sort

81
src/concurrency/parallel/rayon-thumbnails.md Normal file
View file

@ -0,0 +1,81 @@
## Generate jpg thumbnails in parallel
[![rayon-badge]][rayon] [![glob-badge]][glob] [![image-badge]][image] [![cat-concurrency-badge]][cat-concurrency] [![cat-filesystem-badge]][cat-filesystem]
This example generates thumbnails for all .jpg files in the current directory
then saves them in a new folder called `thumbnails`.
[`glob::glob_with`] finds jpeg files in current directory. `rayon` resizes
images in parallel using [`par_iter`] calling [`DynamicImage::resize`].
```rust,no_run
# #[macro_use]
# extern crate error_chain;
extern crate glob;
extern crate image;
extern crate rayon;
use std::path::Path;
use std::fs::create_dir_all;
# use error_chain::ChainedError;
use glob::{glob_with, MatchOptions};
use image::{FilterType, ImageError};
use rayon::prelude::*;
# error_chain! {
# foreign_links {
# Image(ImageError);
# Io(std::io::Error);
# Glob(glob::PatternError);
# }
# }
fn run() -> Result<()> {
let options: MatchOptions = Default::default();
let files: Vec<_> = glob_with("*.jpg", &options)?
.filter_map(|x| x.ok())
.collect();
if files.len() == 0 {
bail!("No .jpg files found in current directory");
}
let thumb_dir = "thumbnails";
create_dir_all(thumb_dir)?;
println!("Saving {} thumbnails into '{}'...", files.len(), thumb_dir);
let image_failures: Vec<_> = files
.par_iter()
.map(|path| {
make_thumbnail(path, thumb_dir, 300)
.map_err(|e| e.chain_err(|| path.display().to_string()))
})
.filter_map(|x| x.err())
.collect();
image_failures.iter().for_each(|x| println!("{}", x.display_chain()));
println!("{} thumbnails saved successfully", files.len() - image_failures.len());
Ok(())
}
fn make_thumbnail<PA, PB>(original: PA, thumb_dir: PB, longest_edge: u32) -> Result<()>
where
PA: AsRef<Path>,
PB: AsRef<Path>,
{
let img = image::open(original.as_ref())?;
let file_path = thumb_dir.as_ref().join(original);
Ok(img.resize(longest_edge, longest_edge, FilterType::Nearest)
.save(file_path)?)
}
#
# quick_main!(run);
```
[`glob::glob_with`]: https://docs.rs/glob/*/glob/fn.glob_with.html
[`par_iter`]: https://docs.rs/rayon/*/rayon/iter/trait.IntoParallelRefIterator.html#tymethod.par_iter
[`DynamicImage::resize`]: https://docs.rs/image/*/image/enum.DynamicImage.html#method.resize

40
src/concurrency/thread/crossbeam-spawn.md Normal file
View file

@ -0,0 +1,40 @@
## Spawn a short-lived thread
[![crossbeam-badge]][crossbeam] [![cat-concurrency-badge]][cat-concurrency]
The example uses the [crossbeam] crate, which provides data structures and functions
for concurrent and parallel programming. [`Scope::spawn`] spawns a new scoped thread that is guaranteed
to terminate before returning from the closure that passed into [`crossbeam::scope`] function, meaning that
you can reference data from the calling function.
This example splits the array in half and performs the work in separate threads.
```rust
extern crate crossbeam;
use std::cmp;
fn main() {
let arr = &[-4, 1, 10, 25];
let max = find_max(arr, 0, arr.len());
assert_eq!(25, max);
}
fn find_max(arr: &[i32], start: usize, end: usize) -> i32 {
const THRESHOLD: usize = 2;
if end - start <= THRESHOLD {
return *arr.iter().max().unwrap();
}
let mid = start + (end - start) / 2;
crossbeam::scope(|scope| {
let left = scope.spawn(|| find_max(arr, start, mid));
let right = scope.spawn(|| find_max(arr, mid, end));
cmp::max(left.join(), right.join())
})
}
```
[`crossbeam::scope`]: https://docs.rs/crossbeam/*/crossbeam/fn.scope.html
[`Scope::spawn`]: https://docs.rs/crossbeam/*/crossbeam/struct.Scope.html#method.spawn

50
src/concurrency/thread/global-mut-state.md Normal file
View file

@ -0,0 +1,50 @@
## Maintain global mutable state
[![lazy_static-badge]][lazy_static] [![cat-rust-patterns-badge]][cat-rust-patterns]
Declare global state using [lazy_static]. [lazy_static]
creates a globally available `static ref` which requires a [`Mutex`]
to allow mutation (also see [`RwLock`]). The [`Mutex`] wrap ensures
the state cannot be simultaneously accessed by multiple threads, preventing
race conditions. A [`MutexGuard`] must be acquired to read or mutate the
value stored in a [`Mutex`].
```rust
# #[macro_use]
# extern crate error_chain;
#[macro_use]
extern crate lazy_static;
use std::sync::Mutex;
#
# error_chain!{ }
lazy_static! {
static ref FRUIT: Mutex<Vec<String>> = Mutex::new(Vec::new());
}
fn insert(fruit: &str) -> Result<()> {
let mut db = FRUIT.lock().map_err(|_| "Failed to acquire MutexGuard")?;
db.push(fruit.to_string());
Ok(())
}
fn run() -> Result<()> {
insert("apple")?;
insert("orange")?;
insert("peach")?;
{
let db = FRUIT.lock().map_err(|_| "Failed to acquire MutexGuard")?;
db.iter().enumerate().for_each(|(i, item)| println!("{}: {}", i, item));
}
insert("grape")?;
Ok(())
}
#
# quick_main!(run);
```
[`Mutex`]: https://doc.rust-lang.org/std/sync/struct.Mutex.html
[`MutexGuard`]: https://doc.rust-lang.org/std/sync/struct.MutexGuard.html
[`RwLock`]: https://doc.rust-lang.org/std/sync/struct.RwLock.html

131
src/concurrency/thread/threadpool-fractal.md Normal file
View file

@ -0,0 +1,131 @@
## Draw fractal dispatching work to a thread pool
[![threadpool-badge]][threadpool] [![num-badge]][num] [![num_cpus-badge]][num_cpus] [![image-badge]][image] [![cat-concurrency-badge]][cat-concurrency][![cat-science-badge]][cat-science][![cat-rendering-badge]][cat-rendering]
This example generates an image by drawing a fractal from the [Julia set]
with a thread pool for distributed computation.
<a href="https://cloud.githubusercontent.com/assets/221000/26546700/9be34e80-446b-11e7-81dc-dd9871614ea1.png"><img src="https://cloud.githubusercontent.com/assets/221000/26546700/9be34e80-446b-11e7-81dc-dd9871614ea1.png" width="150" /></a>
Allocate memory for output image of given width and height with [`ImageBuffer::new`].
[`Rgb::from_channels`] calculates RGB pixel values.
Create [`ThreadPool`] with thread count equal to number of cores with [`num_cpus::get`].
[`ThreadPool::execute`] receives each pixel as a separate job.
[`mpsc::channel`] receives the jobs and [`Receiver::recv`] retrieves them.
[`ImageBuffer::put_pixel`] uses the data to set the pixel color.
[`ImageBuffer::save`] writes the image to `output.png`.
```rust,no_run
# #[macro_use]
# extern crate error_chain;
extern crate threadpool;
extern crate num;
extern crate num_cpus;
extern crate image;
use std::sync::mpsc::{channel, RecvError};
use threadpool::ThreadPool;
use num::complex::Complex;
use image::{ImageBuffer, Pixel, Rgb};
#
# error_chain! {
# foreign_links {
# MpscRecv(RecvError);
# Io(std::io::Error);
# }
# }
#
# // Function converting intensity values to RGB
# // Based on http://www.efg2.com/Lab/ScienceAndEngineering/Spectra.htm
# fn wavelength_to_rgb(wavelength: u32) -> Rgb<u8> {
# let wave = wavelength as f32;
#
# let (r, g, b) = match wavelength {
# 380...439 => ((440. - wave) / (440. - 380.), 0.0, 1.0),
# 440...489 => (0.0, (wave - 440.) / (490. - 440.), 1.0),
# 490...509 => (0.0, 1.0, (510. - wave) / (510. - 490.)),
# 510...579 => ((wave - 510.) / (580. - 510.), 1.0, 0.0),
# 580...644 => (1.0, (645. - wave) / (645. - 580.), 0.0),
# 645...780 => (1.0, 0.0, 0.0),
# _ => (0.0, 0.0, 0.0),
# };
#
# let factor = match wavelength {
# 380...419 => 0.3 + 0.7 * (wave - 380.) / (420. - 380.),
# 701...780 => 0.3 + 0.7 * (780. - wave) / (780. - 700.),
# _ => 1.0,
# };
#
# let (r, g, b) = (normalize(r, factor), normalize(g, factor), normalize(b, factor));
# Rgb::from_channels(r, g, b, 0)
# }
#
# // Maps Julia set distance estimation to intensity values
# fn julia(c: Complex<f32>, x: u32, y: u32, width: u32, height: u32, max_iter: u32) -> u32 {
# let width = width as f32;
# let height = height as f32;
#
# let mut z = Complex {
# // scale and translate the point to image coordinates
# re: 3.0 * (x as f32 - 0.5 * width) / width,
# im: 2.0 * (y as f32 - 0.5 * height) / height,
# };
#
# let mut i = 0;
# for t in 0..max_iter {
# if z.norm() >= 2.0 {
# break;
# }
# z = z * z + c;
# i = t;
# }
# i
# }
#
# // Normalizes color intensity values within RGB range
# fn normalize(color: f32, factor: f32) -> u8 {
# ((color * factor).powf(0.8) * 255.) as u8
# }
fn run() -> Result<()> {
let (width, height) = (1920, 1080);
let mut img = ImageBuffer::new(width, height);
let iterations = 300;
let c = Complex::new(-0.8, 0.156);
let pool = ThreadPool::new(num_cpus::get());
let (tx, rx) = channel();
for y in 0..height {
let tx = tx.clone();
pool.execute(move || for x in 0..width {
let i = julia(c, x, y, width, height, iterations);
let pixel = wavelength_to_rgb(380 + i * 400 / iterations);
tx.send((x, y, pixel)).expect("Could not send data!");
});
}
for _ in 0..(width * height) {
let (x, y, pixel) = rx.recv()?;
img.put_pixel(x, y, pixel);
}
let _ = img.save("output.png")?;
Ok(())
}
#
# quick_main!(run);
```
[`ImageBuffer::new`]: https://docs.rs/image/*/image/struct.ImageBuffer.html#method.new
[`ImageBuffer::put_pixel`]: https://docs.rs/image/*/image/struct.ImageBuffer.html#method.put_pixel
[`ImageBuffer::save`]: https://docs.rs/image/*/image/struct.ImageBuffer.html#method.save
[`mpsc::channel`]: https://doc.rust-lang.org/std/sync/mpsc/fn.channel.html
[`num_cpus::get`]: https://docs.rs/num_cpus/*/num_cpus/fn.get.html
[`Receiver::recv`]: https://doc.rust-lang.org/std/sync/mpsc/struct.Receiver.html#method.recv
[`Rgb::from_channels`]: https://docs.rs/image/*/image/struct.Rgb.html#method.from_channels
[`ThreadPool`]: https://docs.rs/threadpool/*/threadpool/struct.ThreadPool.html
[`ThreadPool::execute`]: https://docs.rs/threadpool/*/threadpool/struct.ThreadPool.html#method.execute
[Julia set]: https://en.wikipedia.org/wiki/Julia_set

88
src/concurrency/thread/threadpool-walk.md Normal file
View file

@ -0,0 +1,88 @@
## Calculate SHA1 sum of iso files concurrently
[![threadpool-badge]][threadpool] [![num_cpus-badge]][num_cpus] [![walkdir-badge]][walkdir] [![ring-badge]][ring] [![cat-concurrency-badge]][cat-concurrency][![cat-filesystem-badge]][cat-filesystem]
This example calculates the SHA1 for every file with iso extension in the
current directory. A threadpool generates threads equal to the number of cores
present in the system found with [`num_cpus::get`]. [`Walkdir::new`] iterates
the current directory and calls [`execute`] to perform the operations of reading
and computing SHA1 hash.
```rust,no_run
# #[macro_use]
# extern crate error_chain;
extern crate walkdir;
extern crate ring;
extern crate num_cpus;
extern crate threadpool;
# error_chain! {
# foreign_links {
# Io(std::io::Error);
# }
# }
#
use walkdir::WalkDir;
use std::fs::File;
use std::io::{BufReader, Read};
use std::path::Path;
use threadpool::ThreadPool;
use std::sync::mpsc::channel;
use ring::digest::{Context, Digest, SHA1};
# // Verify the iso extension
# fn is_iso(entry: &Path) -> bool {
# match entry.extension() {
# Some(e) if e.to_string_lossy().to_lowercase() == "iso" => true,
# _ => false,
# }
# }
#
fn compute_digest<P: AsRef<Path>>(filepath: P) -> Result<(Digest, P)> {
let mut buf_reader = BufReader::new(File::open(&filepath)?);
let mut context = Context::new(&SHA1);
let mut buffer = [0; 1024];
loop {
let count = buf_reader.read(&mut buffer)?;
if count == 0 {
break;
}
context.update(&buffer[..count]);
}
Ok((context.finish(), filepath))
}
fn run() -> Result<()> {
let pool = ThreadPool::new(num_cpus::get());
let (tx, rx) = channel();
for entry in WalkDir::new("/home/user/Downloads")
.follow_links(true)
.into_iter()
.filter_map(|e| e.ok())
.filter(|e| !e.path().is_dir() && is_iso(e.path())) {
let path = entry.path().to_owned();
let tx = tx.clone();
pool.execute(move || {
let digest = compute_digest(path);
tx.send(digest).expect("Could not send data!");
});
}
drop(tx);
for t in rx.iter() {
let (sha, path) = t?;
println!("{:?} {:?}", sha, path);
}
Ok(())
}
#
# quick_main!(run);
```
[`execute`]: https://docs.rs/threadpool/*/threadpool/struct.ThreadPool.html#method.execute
[`num_cpus::get`]: https://docs.rs/num_cpus/*/num_cpus/fn.get.html
[`Walkdir::new`]: https://docs.rs/walkdir/*/walkdir/struct.WalkDir.html#method.new

11
src/concurrency/threads.md Normal file
View file

@ -0,0 +1,11 @@
# Threads
{{#include thread/crossbeam-spawn.md}}
{{#include thread/global-mut-state.md}}
{{#include thread/threadpool-walk.md}}
{{#include thread/threadpool-fractal.md}}
{{#include ../links.md}}

13
src/cryptography.md Normal file
View file

@ -0,0 +1,13 @@
# Cryptography
| Recipe | Crates | Categories |
|--------|--------|------------|
| [Calculate the SHA-256 digest of a file][ex-sha-digest] | [![ring-badge]][ring] [![data-encoding-badge]][data-encoding] | [![cat-cryptography-badge]][cat-cryptography] |
| [Sign and verify a message with an HMAC digest][ex-hmac] | [![ring-badge]][ring] | [![cat-cryptography-badge]][cat-cryptography] |
| [Salt and hash a password with PBKDF2][ex-pbkdf2] | [![ring-badge]][ring] [![data-encoding-badge]][data-encoding] | [![cat-cryptography-badge]][cat-cryptography] |
[ex-sha-digest]: cryptography/hashing.html#calculate-the-sha-256-digest-of-a-file
[ex-hmac]: cryptography/hashing.html#sign-and-verify-a-message-with-hmac-digest
[ex-pbkdf2]: cryptography/encryption.html#salt-and-hash-a-password-with-pbkdf2
{{#include links.md}}

5
src/cryptography/encryption.md Normal file
View file

@ -0,0 +1,5 @@
# Encryption
{{#include encryption/pbkdf2.md}}
{{#include ../links.md}}

76
src/cryptography/encryption/pbkdf2.md Normal file
View file

@ -0,0 +1,76 @@
<a name="ex-pbkdf2"></a>
## Salt and hash a password with PBKDF2
[![ring-badge]][ring] [![data-encoding-badge]][data-encoding] [![cat-cryptography-badge]][cat-cryptography]
Uses [`ring::pbkdf2`] to hash a salted password using the PBKDF2 key derivation
function [`pbkdf2::derive`]. Verifies the hash is correct with
[`pbkdf2::verify`]. The salt is generated using
[`SecureRandom::fill`], which fills the salt byte array with
securely generated random numbers.
```rust
# #[macro_use]
# extern crate error_chain;
extern crate data_encoding;
extern crate ring;
#
# error_chain! {
# foreign_links {
# Ring(ring::error::Unspecified);
# }
# }
use data_encoding::HEXUPPER;
use ring::{digest, pbkdf2, rand};
use ring::rand::SecureRandom;
fn run() -> Result<()> {
const CREDENTIAL_LEN: usize = digest::SHA512_OUTPUT_LEN;
const N_ITER: u32 = 100_000;
let rng = rand::SystemRandom::new();
let mut salt = [0u8; CREDENTIAL_LEN];
rng.fill(&mut salt)?;
let password = "Guess Me If You Can!";
let mut pbkdf2_hash = [0u8; CREDENTIAL_LEN];
pbkdf2::derive(
&digest::SHA512,
N_ITER,
&salt,
password.as_bytes(),
&mut pbkdf2_hash,
);
println!("Salt: {}", HEXUPPER.encode(&salt));
println!("PBKDF2 hash: {}", HEXUPPER.encode(&pbkdf2_hash));
let should_succeed = pbkdf2::verify(
&digest::SHA512,
N_ITER,
&salt,
password.as_bytes(),
&pbkdf2_hash,
);
let wrong_password = "Definitely not the correct password";
let should_fail = pbkdf2::verify(
&digest::SHA512,
N_ITER,
&salt,
wrong_password.as_bytes(),
&pbkdf2_hash,
);
assert!(should_succeed.is_ok());
assert!(!should_fail.is_ok());
Ok(())
}
#
# quick_main!(run);
```
[`pbkdf2::derive`]: https://briansmith.org/rustdoc/ring/pbkdf2/fn.derive.html
[`pbkdf2::verify`]: https://briansmith.org/rustdoc/ring/pbkdf2/fn.verify.html
[`ring::pbkdf2`]: https://briansmith.org/rustdoc/ring/pbkdf2/index.html
[`SecureRandom::fill`]: https://briansmith.org/rustdoc/ring/rand/trait.SecureRandom.html#tymethod.fill

7
src/cryptography/hashing.md Normal file
View file

@ -0,0 +1,7 @@
# Hashing
{{#include hashing/sha-digest.md}}
{{#include hashing/hmac.md}}
{{#include ../links.md}}

38
src/cryptography/hashing/hmac.md Normal file
View file

@ -0,0 +1,38 @@
## Sign and verify a message with HMAC digest
[![ring-badge]][ring] [![cat-cryptography-badge]][cat-cryptography]
Uses [`ring::hmac`] to creates a [`hmac::Signature`] of a string then verifies the signature is correct.
```rust
# #[macro_use]
# extern crate error_chain;
extern crate ring;
#
# error_chain! {
# foreign_links {
# Ring(ring::error::Unspecified);
# }
# }
use ring::{digest, hmac, rand};
use ring::rand::SecureRandom;
fn run() -> Result<()> {
let mut key_value = [0u8; 48];
let rng = rand::SystemRandom::new();
rng.fill(&mut key_value)?;
let key = hmac::SigningKey::new(&digest::SHA256, &key_value);
let message = "Legitimate and important message.";
let signature = hmac::sign(&key, message.as_bytes());
hmac::verify_with_own_key(&key, message.as_bytes(), signature.as_ref())?;
Ok(())
}
#
# quick_main!(run);
```
[`hmac::Signature`]: https://briansmith.org/rustdoc/ring/hmac/struct.Signature.html
[`ring::hmac`]: https://briansmith.org/rustdoc/ring/hmac/

60
src/cryptography/hashing/sha-digest.md Normal file
View file

@ -0,0 +1,60 @@
## Calculate the SHA-256 digest of a file
[![ring-badge]][ring] [![data-encoding-badge]][data-encoding] [![cat-cryptography-badge]][cat-cryptography]
Writes some data to a file, then calculates the SHA-256 [`digest::Digest`] of
the file's contents using [`digest::Context`].
```rust
# #[macro_use]
# extern crate error_chain;
extern crate data_encoding;
extern crate ring;
use data_encoding::HEXUPPER;
use ring::digest::{Context, Digest, SHA256};
use std::fs::File;
use std::io::{BufReader, Read, Write};
#
# error_chain! {
# foreign_links {
# Io(std::io::Error);
# Decode(data_encoding::DecodeError);
# }
# }
fn sha256_digest<R: Read>(mut reader: R) -> Result<Digest> {
let mut context = Context::new(&SHA256);
let mut buffer = [0; 1024];
loop {
let count = reader.read(&mut buffer)?;
if count == 0 {
break;
}
context.update(&buffer[..count]);
}
Ok(context.finish())
}
fn run() -> Result<()> {
let path = "file.txt";
let mut output = File::create(path)?;
write!(output, "We will generate a digest of this text")?;
let input = File::open(path)?;
let reader = BufReader::new(input);
let digest = sha256_digest(reader)?;
println!("SHA-256 digest is {}", HEXUPPER.encode(digest.as_ref()));
Ok(())
}
#
# quick_main!(run);
```
[`digest::Context`]: https://briansmith.org/rustdoc/ring/digest/struct.Context.html
[`digest::Digest`]: https://briansmith.org/rustdoc/ring/digest/struct.Digest.html

9
src/data_structures.md Normal file
View file

@ -0,0 +1,9 @@
# Data Structures
| Recipe | Crates | Categories |
|--------|--------|------------|
| [Define and operate on a type represented as a bitfield][ex-bitflags] | [![bitflags-badge]][bitflags] | [![cat-no-std-badge]][cat-no-std] |
[ex-bitflags]: data_structures/bitfield.html#define-and-operate-on-a-type-represented-as-a-bitfield
{{#include links.md}}

5
src/data_structures/bitfield.md Normal file
View file

@ -0,0 +1,5 @@
# Custom
{{#include bitfield/bitfield.md}}
{{#include ../links.md}}

56
src/data_structures/bitfield/bitfield.md Normal file
View file

@ -0,0 +1,56 @@
## Define and operate on a type represented as a bitfield
[![bitflags-badge]][bitflags] [![cat-no-std-badge]][cat-no-std]
Creates type safe bitfield type `MyFlags` with help of [`bitflags!`] macro
and implements elementary `clear` operation as well as [`Display`] trait for it.
Subsequently, shows basic bitwise operations and formatting.
```rust
#[macro_use]
extern crate bitflags;
use std::fmt;
bitflags! {
struct MyFlags: u32 {
const FLAG_A = 0b00000001;
const FLAG_B = 0b00000010;
const FLAG_C = 0b00000100;
const FLAG_ABC = Self::FLAG_A.bits
| Self::FLAG_B.bits
| Self::FLAG_C.bits;
}
}
impl MyFlags {
pub fn clear(&mut self) -> &mut MyFlags {
self.bits = 0;
self
}
}
impl fmt::Display for MyFlags {
fn fmt(&self, f: &mut fmt::Formatter) -> fmt::Result {
write!(f, "{:032b}", self.bits)
}
}
fn main() {
let e1 = MyFlags::FLAG_A | MyFlags::FLAG_C;
let e2 = MyFlags::FLAG_B | MyFlags::FLAG_C;
assert_eq!((e1 | e2), MyFlags::FLAG_ABC);
assert_eq!((e1 & e2), MyFlags::FLAG_C);
assert_eq!((e1 - e2), MyFlags::FLAG_A);
assert_eq!(!e2, MyFlags::FLAG_A);
let mut flags = MyFlags::FLAG_ABC;
assert_eq!(format!("{}", flags), "00000000000000000000000000000111");
assert_eq!(format!("{}", flags.clear()), "00000000000000000000000000000000");
assert_eq!(format!("{:?}", MyFlags::FLAG_B), "FLAG_B");
assert_eq!(format!("{:?}", MyFlags::FLAG_A | MyFlags::FLAG_B), "FLAG_A | FLAG_B");
}
```
[`bitflags!`]: https://docs.rs/bitflags/*/bitflags/macro.bitflags.html
[`Display`]: https://doc.rust-lang.org/std/fmt/trait.Display.html

21
src/datetime.md Normal file
View file

@ -0,0 +1,21 @@
# Date and Time
| Recipe | Crates | Categories |
|--------|--------|------------|
| [Measure elapsed time][ex-measure-elapsed-time] | [![std-badge]][std] | [![cat-time-badge]][cat-time] |
| [Perform checked date and time calculations][ex-datetime-arithmetic] | [![chrono-badge]][chrono] | [![cat-date-and-time-badge]][cat-date-and-time] |
| [Convert a local time to another timezone][ex-convert-datetime-timezone] | [![chrono-badge]][chrono] | [![cat-date-and-time-badge]][cat-date-and-time] |
| [Examine the date and time][ex-examine-date-and-time] | [![chrono-badge]][chrono] | [![cat-date-and-time-badge]][cat-date-and-time] |
| [Convert date to UNIX timestamp and vice versa][ex-convert-datetime-timestamp] | [![chrono-badge]][chrono] | [![cat-date-and-time-badge]][cat-date-and-time] |
| [Display formatted date and time][ex-format-datetime] | [![chrono-badge]][chrono] | [![cat-date-and-time-badge]][cat-date-and-time] |
| [Parse string into DateTime struct][ex-parse-datetime] | [![chrono-badge]][chrono] | [![cat-date-and-time-badge]][cat-date-and-time] |
[ex-measure-elapsed-time]: datetime/duration.html#measure-the-elapsed-time-between-two-code-sections
[ex-datetime-arithmetic]: datetime/duration.html#perform-checked-date-and-time-calculations
[ex-convert-datetime-timezone]: datetime/duration.html#convert-a-local-time-to-another-timezone
[ex-examine-date-and-time]: datetime/parse.html#examine-the-date-and-time
[ex-convert-datetime-timestamp]: datetime/parse.html#convert-date-to-unix-timestamp-and-vice-versa
[ex-format-datetime]: datetime/parse.html#display-formatted-date-and-time
[ex-parse-datetime]: datetime/parse.html#parse-string-into-datetime-struct
{{#include links.md}}

9
src/datetime/duration.md Normal file
View file

@ -0,0 +1,9 @@
# Duration and Calculation
{{#include duration/profile.md}}
{{#include duration/checked.md}}
{{#include duration/timezone.md}}
{{#include ../links.md}}

44
src/datetime/duration/checked.md Normal file
View file

@ -0,0 +1,44 @@
## Perform checked date and time calculations
[![chrono-badge]][chrono] [![cat-date-and-time-badge]][cat-date-and-time]
Calculates and displays the date and time two weeks from now using
[`DateTime::checked_add_signed`] and the date of the day before that using
[`DateTime::checked_sub_signed`]. The methods return None if the date and time
cannot be calculated.
Escape sequences that are available for the
[`DateTime::format`] can be found at [`chrono::format::strftime`].
```rust
extern crate chrono;
use chrono::{DateTime, Duration, Utc};
fn day_earlier(date_time: DateTime<Utc>) -> Option<DateTime<Utc>> {
date_time.checked_sub_signed(Duration::days(1))
}
fn main() {
let now = Utc::now();
println!("{}", now);
let almost_three_weeks_from_now = now.checked_add_signed(Duration::weeks(2))
.and_then(|in_2weeks| in_2weeks.checked_add_signed(Duration::weeks(1)))
.and_then(day_earlier);
match almost_three_weeks_from_now {
Some(x) => println!("{}", x),
None => eprintln!("Almost three weeks from now overflows!"),
}
match now.checked_add_signed(Duration::max_value()) {
Some(x) => println!("{}", x),
None => eprintln!("We can't use chrono to tell the time for the Solar System to complete more than one full orbit around the galactic center."),
}
}
```
[`chrono::format::strftime`]: https://docs.rs/chrono/*/chrono/format/strftime/index.html
[`DateTime::checked_add_signed`]: https://docs.rs/chrono/*/chrono/struct.Date.html#method.checked_add_signed
[`DateTime::checked_sub_signed`]: https://docs.rs/chrono/*/chrono/struct.Date.html#method.checked_sub_signed
[`DateTime::format`]: https://docs.rs/chrono/*/chrono/struct.DateTime.html#method.format

30
src/datetime/duration/profile.md Normal file
View file

@ -0,0 +1,30 @@
## Measure the elapsed time between two code sections
[![std-badge]][std] [![cat-time-badge]][cat-time]
Measures [`time::Instant::elapsed`] since [`time::Instant::now`].
Calling [`time::Instant::elapsed`] returns a [`time::Duration`] that we print at the end of the example.
This method will not mutate or reset the [`time::Instant`] object.
```rust
use std::time::{Duration, Instant};
# use std::thread;
#
# fn expensive_function() {
# thread::sleep(Duration::from_secs(1));
# }
fn main() {
let start = Instant::now();
expensive_function();
let duration = start.elapsed();
println!("Time elapsed in expensive_function() is: {:?}", duration);
}
```
[`time::Duration`]: https://doc.rust-lang.org/std/time/struct.Duration.html
[`time::Instant::elapsed`]: https://doc.rust-lang.org/std/time/struct.Instant.html#method.elapsed
[`time::Instant::now`]: https://doc.rust-lang.org/std/time/struct.Instant.html#method.now
[`time::Instant`]:https://doc.rust-lang.org/std/time/struct.Instant.html

29
src/datetime/duration/timezone.md Normal file
View file

@ -0,0 +1,29 @@
## Convert a local time to another timezone
[![chrono-badge]][chrono] [![cat-date-and-time-badge]][cat-date-and-time]
Gets the local time and displays it using [`offset::Local::now`] and then converts it to the UTC standard using the [`DateTime::from_utc`] struct method. A time is then converted using the [`offset::FixedOffset`] struct and the UTC time is then converted to UTC+8 and UTC-2.
```rust
extern crate chrono;
use chrono::{DateTime, FixedOffset, Local, Utc};
fn main() {
let local_time = Local::now();
let utc_time = DateTime::<Utc>::from_utc(local_time.naive_utc(), Utc);
let china_timezone = FixedOffset::east(8 * 3600);
let rio_timezone = FixedOffset::west(2 * 3600);
println!("Local time now is {}", local_time);
println!("UTC time now is {}", utc_time);
println!(
"Time in Hong Kong now is {}",
utc_time.with_timezone(&china_timezone)
);
println!("Time in Rio de Janeiro now is {}", utc_time.with_timezone(&rio_timezone));
}
```
[`DateTime::from_utc`]:https://docs.rs/chrono/*/chrono/struct.DateTime.html#method.from_utc
[`offset::FixedOffset`]: https://docs.rs/chrono/*/chrono/offset/struct.FixedOffset.html
[`offset::Local::now`]: https://docs.rs/chrono/*/chrono/offset/struct.Local.html#method.now

11
src/datetime/parse.md Normal file
View file

@ -0,0 +1,11 @@
# Parsing and Displaying
{{#include parse/current.md}}
{{#include parse/timestamp.md}}
{{#include parse/format.md}}
{{#include parse/string.md}}
{{#include ../links.md}}

46
src/datetime/parse/current.md Normal file
View file

@ -0,0 +1,46 @@
## Examine the date and time
[![chrono-badge]][chrono] [![cat-date-and-time-badge]][cat-date-and-time]
Gets the current UTC [`DateTime`] and its hour/minute/second via [`Timelike`]
and its year/month/day/weekday via [`Datelike`].
```rust
extern crate chrono;
use chrono::{Datelike, Timelike, Utc};
fn main() {
let now = Utc::now();
let (is_pm, hour) = now.hour12();
println!(
"The current UTC time is {:02}:{:02}:{:02} {}",
hour,
now.minute(),
now.second(),
if is_pm { "PM" } else { "AM" }
);
println!(
"And there have been {} seconds since midnight",
now.num_seconds_from_midnight()
);
let (is_common_era, year) = now.year_ce();
println!(
"The current UTC date is {}-{:02}-{:02} {:?} ({})",
year,
now.month(),
now.day(),
now.weekday(),
if is_common_era { "CE" } else { "BCE" }
);
println!(
"And the Common Era began {} days ago",
now.num_days_from_ce()
);
}
```
[`Datelike`]: https://docs.rs/chrono/*/chrono/trait.Datelike.html
[`DateTime`]: https://docs.rs/chrono/*/chrono/struct.DateTime.html
[`Timelike`]: https://docs.rs/chrono/*/chrono/trait.Timelike.html

30
src/datetime/parse/format.md Normal file
View file

@ -0,0 +1,30 @@
## Display formatted date and time
[![chrono-badge]][chrono] [![cat-date-and-time-badge]][cat-date-and-time]
Gets and displays the current time in UTC using [`Utc::now`]. Formats the
current time in the well-known formats [RFC 2822] using [`DateTime::to_rfc2822`]
and [RFC 3339] using [`DateTime::to_rfc3339`], and in a custom format using
[`DateTime::format`].
```rust
extern crate chrono;
use chrono::{DateTime, Utc};
fn main() {
let now: DateTime<Utc> = Utc::now();
println!("UTC now is: {}", now);
println!("UTC now in RFC 2822 is: {}", now.to_rfc2822());
println!("UTC now in RFC 3339 is: {}", now.to_rfc3339());
println!("UTC now in a custom format is: {}", now.format("%a %b %e %T %Y"));
}
```
[`DateTime::format`]: https://docs.rs/chrono/*/chrono/struct.DateTime.html#method.format
[`DateTime::to_rfc2822`]: https://docs.rs/chrono/*/chrono/struct.DateTime.html#method.to_rfc2822
[`DateTime::to_rfc3339`]: https://docs.rs/chrono/*/chrono/struct.DateTime.html#method.to_rfc3339
[`Utc::now`]: https://docs.rs/chrono/*/chrono/offset/struct.Utc.html#method.now
[RFC 2822]: https://www.ietf.org/rfc/rfc2822.txt
[RFC 3339]: https://www.ietf.org/rfc/rfc3339.txt

67
src/datetime/parse/string.md Normal file
View file

@ -0,0 +1,67 @@
## Parse string into DateTime struct
[![chrono-badge]][chrono] [![cat-date-and-time-badge]][cat-date-and-time]
Parses a [`DateTime`] struct from strings representing the well-known formats
[RFC 2822], [RFC 3339], and a custom format, using
[`DateTime::parse_from_rfc2822`], [`DateTime::parse_from_rfc3339`], and
[`DateTime::parse_from_str`] respectively.
Escape sequences that are available for the [`DateTime::parse_from_str`] can be
found at [`chrono::format::strftime`]. Note that the [`DateTime::parse_from_str`]
requires that such a DateTime struct must be creatable that it uniquely
identifies a date and a time. For parsing dates and times without timezones use
[`NaiveDate`], [`NaiveTime`], and [`NaiveDateTime`].
```rust
extern crate chrono;
# #[macro_use]
# extern crate error_chain;
#
use chrono::{DateTime, NaiveDate, NaiveDateTime, NaiveTime};
#
# error_chain! {
# foreign_links {
# DateParse(chrono::format::ParseError);
# }
# }
fn run() -> Result<()> {
let rfc2822 = DateTime::parse_from_rfc2822("Tue, 1 Jul 2003 10:52:37 +0200")?;
println!("{}", rfc2822);
let rfc3339 = DateTime::parse_from_rfc3339("1996-12-19T16:39:57-08:00")?;
println!("{}", rfc3339);
let custom = DateTime::parse_from_str("5.8.1994 8:00 am +0000", "%d.%m.%Y %H:%M %P %z")?;
println!("{}", custom);
let time_only = NaiveTime::parse_from_str("23:56:04", "%H:%M:%S")?;
println!("{}", time_only);
let date_only = NaiveDate::parse_from_str("2015-09-05", "%Y-%m-%d")?;
println!("{}", date_only);
let no_timezone = NaiveDateTime::parse_from_str("2015-09-05 23:56:04", "%Y-%m-%d %H:%M:%S")?;
println!("{}", no_timezone);
Ok(())
}
#
# quick_main!(run);
```
[`chrono::format::strftime`]: https://docs.rs/chrono/*/chrono/format/strftime/index.html
[`DateTime::format`]: https://docs.rs/chrono/*/chrono/struct.DateTime.html#method.format
[`DateTime::parse_from_rfc2822`]: https://docs.rs/chrono/*/chrono/struct.DateTime.html#method.parse_from_rfc2822
[`DateTime::parse_from_rfc3339`]: https://docs.rs/chrono/*/chrono/struct.DateTime.html#method.parse_from_rfc3339
[`DateTime::parse_from_str`]: https://docs.rs/chrono/*/chrono/struct.DateTime.html#method.parse_from_str
[`DateTime::to_rfc2822`]: https://docs.rs/chrono/*/chrono/struct.DateTime.html#method.to_rfc2822
[`DateTime::to_rfc3339`]: https://docs.rs/chrono/*/chrono/struct.DateTime.html#method.to_rfc3339
[`DateTime`]: https://docs.rs/chrono/*/chrono/struct.DateTime.html
[`NaiveDate`]: https://docs.rs/chrono/*/chrono/naive/struct.NaiveDate.html
[`NaiveDateTime`]: https://docs.rs/chrono/*/chrono/naive/struct.NaiveDateTime.html
[`NaiveTime`]: https://docs.rs/chrono/*/chrono/naive/struct.NaiveTime.html
[RFC 2822]: https://www.ietf.org/rfc/rfc2822.txt
[RFC 3339]: https://www.ietf.org/rfc/rfc3339.txt

32
src/datetime/parse/timestamp.md Normal file
View file

@ -0,0 +1,32 @@
## Convert date to UNIX timestamp and vice versa
[![chrono-badge]][chrono] [![cat-date-and-time-badge]][cat-date-and-time]
Converts a date given by [`NaiveDate::from_ymd`] and [`NaiveTime::from_hms`]
to [UNIX timestamp] using [`NaiveDateTime::timestamp`].
Then it calculates what was the date after one billion seconds
since January 1, 1970 0:00:00 UTC, using [`NaiveDateTime::from_timestamp`].
```rust
extern crate chrono;
use chrono::{NaiveDate, NaiveDateTime};
fn main() {
let date_time: NaiveDateTime = NaiveDate::from_ymd(2017, 11, 12).and_hms(17, 33, 44);
println!(
"Number of seconds between 1970-01-01 00:00:00 and {} is {}.",
date_time, date_time.timestamp());
let date_time_after_a_billion_seconds = NaiveDateTime::from_timestamp(1_000_000_000, 0);
println!(
"Date after a billion seconds since 1970-01-01 00:00:00 was {}.",
date_time_after_a_billion_seconds);
}
```
[`NaiveDate::from_ymd`]: https://docs.rs/chrono/*/chrono/naive/struct.NaiveDate.html#method.from_ymd
[`NaiveDateTime::from_timestamp`]: https://docs.rs/chrono/*/chrono/naive/struct.NaiveDateTime.html#method.from_timestamp
[`NaiveDateTime::timestamp`]: https://docs.rs/chrono/*/chrono/naive/struct.NaiveDateTime.html#method.timestamp
[`NaiveTime::from_hms`]: https://docs.rs/chrono/*/chrono/naive/struct.NaiveTime.html#method.from_hms
[UNIX timestamp]: https://en.wikipedia.org/wiki/Unix_time

33
src/development_tools.md Normal file
View file

@ -0,0 +1,33 @@
# Development Tools
{{#include development_tools/debugging.md}}
## Versioning
| Recipe | Crates | Categories |
|--------|--------|------------|
| [Parse and increment a version string][ex-semver-increment] | [![semver-badge]][semver] | [![cat-config-badge]][cat-config] |
| [Parse a complex version string][ex-semver-complex] | [![semver-badge]][semver] | [![cat-config-badge]][cat-config] |
| [Check if given version is pre-release][ex-semver-prerelease] | [![semver-badge]][semver] | [![cat-config-badge]][cat-config] |
| [Find the latest version satisfying given range][ex-semver-latest] | [![semver-badge]][semver] | [![cat-config-badge]][cat-config] |
| [Check external command version for compatibility][ex-semver-command] | [![semver-badge]][semver] | [![cat-text-processing-badge]][cat-text-processing] [![cat-os-badge]][cat-os]
## Build Time
| Recipe | Crates | Categories |
|--------|--------|------------|
| [Compile and link statically to a bundled C library][ex-cc-static-bundled] | [![cc-badge]][cc] | [![cat-development-tools-badge]][cat-development-tools] |
| [Compile and link statically to a bundled C++ library][ex-cc-static-bundled-cpp] | [![cc-badge]][cc] | [![cat-development-tools-badge]][cat-development-tools] |
| [Compile a C library while setting custom defines][ex-cc-custom-defines] | [![cc-badge]][cc] | [![cat-development-tools-badge]][cat-development-tools] |
[ex-semver-increment]: development_tools/versioning.html#parse-and-increment-a-version-string
[ex-semver-complex]: development_tools/versioning.html#parse-a-complex-version-string
[ex-semver-prerelease]: development_tools/versioning.html#check-if-given-version-is-pre-release
[ex-semver-latest]: development_tools/versioning.html#find-the-latest-version-satisfying-given-range
[ex-semver-command]: development_tools/versioning.html#check-external-command-version-for-compatibility
[ex-cc-static-bundled]: development_tools/build_tools.html#compile-and-link-statically-to-a-bundled-c-library
[ex-cc-static-bundled-cpp]: development_tools/build_tools.html#compile-and-link-statically-to-a-bundled-c-library-1
[ex-cc-custom-defines]: development_tools/build_tools.html#compile-a-c-library-while-setting-custom-defines
{{#include links.md}}

16
src/development_tools/build_tools.md Normal file
View file

@ -0,0 +1,16 @@
# Build Time Tooling
This section covers "build-time" tooling, or code that is run prior to compiling a crate's source code.
Conventionally, build-time code lives in a **build.rs** file and is commonly referred to as a "build script".
Common use cases include rust code generation and compilation of bundled C/C++/asm code.
See crates.io's [documentation on the matter][build-script-docs] for more information.
{{#include build_tools/cc-bundled-static.md}}
{{#include build_tools/cc-bundled-cpp.md}}
{{#include build_tools/cc-defines.md}}
{{#include ../links.md}}
[build-script-docs]: http://doc.crates.io/build-script.html

58
src/development_tools/build_tools/cc-bundled-cpp.md Normal file
View file

@ -0,0 +1,58 @@
## Compile and link statically to a bundled C++ library
[![cc-badge]][cc] [![cat-development-tools-badge]][cat-development-tools]
Linking a bundled C++ library is very similar to linking a bundled C library. The two core differences when compiling and statically linking a bundled C++ library are specifying a C++ compiler via the builder method [`cpp(true)`][cc-build-cpp] and preventing name mangling by the C++ compiler by adding the `extern "C"` section at the top of our C++ source file.
### `Cargo.toml`
```toml
[package]
...
build = "build.rs"
[build-dependencies]
cc = "1"
```
### `build.rs`
```rust,no_run
extern crate cc;
fn main() {
cc::Build::new()
.cpp(true)
.file("src/foo.cpp")
.compile("foo");
}
```
### `src/foo.cpp`
```cpp
extern "C" {
int multiply(int x, int y);
}
int multiply(int x, int y) {
return x*y;
}
```
### `src/main.rs`
```rust,ignore
extern {
fn multiply(x : i32, y : i32) -> i32;
}
fn main(){
unsafe {
println!("{}", multiply(5,7));
}
}
```
[cc-build-cpp]: https://docs.rs/cc/*/cc/struct.Build.html#method.cpp

103
src/development_tools/build_tools/cc-bundled-static.md Normal file
View file

@ -0,0 +1,103 @@
## Compile and link statically to a bundled C library
[![cc-badge]][cc] [![cat-development-tools-badge]][cat-development-tools]
To accommodate scenarios where additional C, C++, or assembly is required in a project, the [**cc**][cc] crate
offers a simple api for compiling bundled C/C++/asm code into static libraries (**.a**) that can be statically linked to by **rustc**.
The following example has some bundled C code (**src/hello.c**) that will be used from rust.
Before compiling rust source code, the "build" file (**build.rs**) specified in **Cargo.toml** runs.
Using the [**cc**][cc] crate, a static library file will be produced (in this case, **libhello.a**, see
[`compile` docs][cc-build-compile]) which can then be used from rust by declaring the external function signatures in an `extern` block.
Since the bundled C is very simple, only a single source file needs to be passed to [`cc::Build`][cc-build].
For more complex build requirements, [`cc::Build`][cc-build] offers a full suite of builder methods for specifying
[`include`][cc-build-include] paths and extra compiler [`flag`][cc-build-flag]s.
### `Cargo.toml`
```toml
[package]
...
build = "build.rs"
[build-dependencies]
cc = "1"
[dependencies]
error-chain = "0.11"
```
### `build.rs`
```rust,no_run
extern crate cc;
fn main() {
cc::Build::new()
.file("src/hello.c")
.compile("hello"); // outputs `libhello.a`
}
```
### `src/hello.c`
```c
#include <stdio.h>
void hello() {
printf("Hello from C!\n");
}
void greet(const char* name) {
printf("Hello, %s!\n", name);
}
```
### `src/main.rs`
```rust,ignore
# #[macro_use] extern crate error_chain;
use std::ffi::CString;
use std::os::raw::c_char;
#
# error_chain! {
# foreign_links {
# NulError(::std::ffi::NulError);
# Io(::std::io::Error);
# }
# }
#
# fn prompt(s: &str) -> Result<String> {
# use std::io::Write;
# print!("{}", s);
# std::io::stdout().flush()?;
# let mut input = String::new();
# std::io::stdin().read_line(&mut input)?;
# Ok(input.trim().to_string())
# }
extern {
fn hello();
fn greet(name: *const c_char);
}
fn run() -> Result<()> {
unsafe { hello() }
let name = prompt("What's your name? ")?;
let c_name = CString::new(name)?;
unsafe { greet(c_name.as_ptr()) }
Ok(())
}
#
# quick_main!(run);
```
[`cc::Build::define`]: https://docs.rs/cc/*/cc/struct.Build.html#method.define
[`Option`]: https://doc.rust-lang.org/std/option/enum.Option.html
[cc-build-compile]: https://docs.rs/cc/*/cc/struct.Build.html#method.compile
[cc-build-cpp]: https://docs.rs/cc/*/cc/struct.Build.html#method.cpp
[cc-build-flag]: https://docs.rs/cc/*/cc/struct.Build.html#method.flag
[cc-build-include]: https://docs.rs/cc/*/cc/struct.Build.html#method.include
[cc-build]: https://docs.rs/cc/*/cc/struct.Build.html

66
src/development_tools/build_tools/cc-defines.md Normal file
View file

@ -0,0 +1,66 @@
## Compile a C library while setting custom defines
[![cc-badge]][cc] [![cat-development-tools-badge]][cat-development-tools]
It is simple to build bundled C code with custom defines using [`cc::Build::define`].
The method takes an [`Option`] value, so it is possible to create defines such as `#define APP_NAME "foo"`
as well as `#define WELCOME` (pass `None` as the value for a value-less define). This example builds
a bundled C file with dynamic defines set in `build.rs` and prints "**Welcome to foo - version 1.0.2**"
when run. Cargo sets some [environment variables][cargo-env] which may be useful for some custom defines.
### `Cargo.toml`
```toml
[package]
...
version = "1.0.2"
build = "build.rs"
[build-dependencies]
cc = "1"
```
### `build.rs`
```rust,no_run
extern crate cc;
fn main() {
cc::Build::new()
.define("APP_NAME", "\"foo\"")
.define("VERSION", format!("\"{}\"", env!("CARGO_PKG_VERSION")).as_str())
.define("WELCOME", None)
.file("src/foo.c")
.compile("foo");
}
```
### `src/foo.c`
```c
#include <stdio.h>
void print_app_info() {
#ifdef WELCOME
printf("Welcome to ");
#endif
printf("%s - version %s\n", APP_NAME, VERSION);
}
```
### `src/main.rs`
```rust,ignore
extern {
fn print_app_info();
}
fn main(){
unsafe {
print_app_info();
}
}
```
[cargo-env]: https://doc.rust-lang.org/cargo/reference/environment-variables.html

25
src/development_tools/debugging.md Normal file
View file

@ -0,0 +1,25 @@
## Debugging
| Recipe | Crates | Categories |
|--------|--------|------------|
| [Log a debug message to the console][ex-log-debug] | [![log-badge]][log] [![env_logger-badge]][env_logger] | [![cat-debugging-badge]][cat-debugging] |
| [Log an error message to the console][ex-log-error] | [![log-badge]][log] [![env_logger-badge]][env_logger] | [![cat-debugging-badge]][cat-debugging] |
| [Log to stdout instead of stderr][ex-log-stdout] | [![log-badge]][log] [![env_logger-badge]][env_logger] | [![cat-debugging-badge]][cat-debugging] |
| [Log messages with a custom logger][ex-log-custom-logger] | [![log-badge]][log] | [![cat-debugging-badge]][cat-debugging] |
| [Log to the Unix syslog][ex-log-syslog] | [![log-badge]][log] [![syslog-badge]][syslog] | [![cat-debugging-badge]][cat-debugging] |
| [Enable log levels per module][ex-log-mod] | [![log-badge]][log] [![env_logger-badge]][env_logger] | [![cat-debugging-badge]][cat-debugging] |
| [Use a custom environment variable to set up logging][ex-log-env-variable] | [![log-badge]][log] [![env_logger-badge]][env_logger] | [![cat-debugging-badge]][cat-debugging] |
| [Include timestamp in log messages][ex-log-timestamp] | [![log-badge]][log] [![env_logger-badge]][env_logger] [![chrono-badge]][chrono] | [![cat-debugging-badge]][cat-debugging] |
| [Log messages to a custom location][ex-log-custom] | [![log-badge]][log] [![log4rs-badge]][log4rs] | [![cat-debugging-badge]][cat-debugging] |
[ex-log-debug]: development_tools/debugging/log.html#log-a-debug-message-to-the-console
[ex-log-error]: development_tools/debugging/log.html#log-an-error-message-to-the-console
[ex-log-stdout]: development_tools/debugging/log.html#log-to-stdout-instead-of-stderr
[ex-log-custom-logger]: development_tools/debugging/log.html#log-messages-with-a-custom-logger
[ex-log-syslog]: development_tools/debugging/log.html#log-to-the-unix-syslog
[ex-log-mod]: development_tools/debugging/config_log.html#enable-log-levels-per-module
[ex-log-env-variable]: development_tools/debugging/config_log.html#use-a-custom-environment-variable-to-set-up-logging
[ex-log-timestamp]: development_tools/debugging/config_log.html#include-timestamp-in-log-messages
[ex-log-custom]: development_tools/debugging/config_log.html#log-messages-to-a-custom-location
{{#include ../links.md}}

11
src/development_tools/debugging/config_log.md Normal file
View file

@ -0,0 +1,11 @@
# Configure Logging
{{#include config_log/log-mod.md}}
{{#include config_log/log-env-variable.md}}
{{#include config_log/log-timestamp.md}}
{{#include config_log/log-custom.md}}
{{#include ../../links.md}}

58
src/development_tools/debugging/config_log/log-custom.md Normal file
View file

@ -0,0 +1,58 @@
## Log messages to a custom location
[![log-badge]][log] [![log4rs-badge]][log4rs] [![cat-debugging-badge]][cat-debugging]
[log4rs] configures log output to a custom location. [log4rs] can use either an
external YAML file or a builder configuration.
Create the log configuration with [`log4rs::append::file::FileAppender`]. An
appender defines the logging destination. The configuration continues with
encoding using a custom pattern from [`log4rs::encode::pattern`].
Assigns the configuration to [`log4rs::config::Config`] and sets the default
[`log::LevelFilter`].
```rust,no_run
# #[macro_use]
# extern crate error_chain;
#[macro_use]
extern crate log;
extern crate log4rs;
use log::LevelFilter;
use log4rs::append::file::FileAppender;
use log4rs::encode::pattern::PatternEncoder;
use log4rs::config::{Appender, Config, Root};
#
# error_chain! {
# foreign_links {
# Io(std::io::Error);
# LogConfig(log4rs::config::Errors);
# SetLogger(log::SetLoggerError);
# }
# }
fn run() -> Result<()> {
let logfile = FileAppender::builder()
.encoder(Box::new(PatternEncoder::new("{l} - {m}\n")))
.build("log/output.log")?;
let config = Config::builder()
.appender(Appender::builder().build("logfile", Box::new(logfile)))
.build(Root::builder()
.appender("logfile")
.build(LevelFilter::Info))?;
log4rs::init_config(config)?;
info!("Hello, world!");
Ok(())
}
#
# quick_main!(run);
```
[`log4rs::append::file::FileAppender`]: https://docs.rs/log4rs/*/log4rs/append/file/struct.FileAppender.html
[`log4rs::config::Config`]: https://docs.rs/log4rs/*/log4rs/config/struct.Config.html
[`log4rs::encode::pattern`]: https://docs.rs/log4rs/*/log4rs/encode/pattern/index.html
[`log::LevelFilter`]: https://docs.rs/log/*/log/enum.LevelFilter.html

35
src/development_tools/debugging/config_log/log-env-variable.md Normal file
View file

@ -0,0 +1,35 @@
## Use a custom environment variable to set up logging
[![log-badge]][log] [![env_logger-badge]][env_logger] [![cat-debugging-badge]][cat-debugging]
[`Builder`] configures logging.
[`Builder::parse`] parses `MY_APP_LOG`
environment variable contents in the form of [`RUST_LOG`] syntax.
Then, [`Builder::init`] initializes the logger.
All these steps are normally done internally by [`env_logger::init`].
```rust
#[macro_use]
extern crate log;
extern crate env_logger;
use std::env;
use env_logger::Builder;
fn main() {
Builder::new()
.parse(&env::var("MY_APP_LOG").unwrap_or_default())
.init();
info!("informational message");
warn!("warning message");
error!("this is an error {}", "message");
}
```
[`env_logger::init`]: https://docs.rs/env_logger/*/env_logger/fn.init.html
[`Builder`]: https://docs.rs/env_logger/*/env_logger/struct.Builder.html
[`Builder::init`]: https://docs.rs/env_logger/*/env_logger/struct.Builder.html#method.init
[`Builder::parse`]: https://docs.rs/env_logger/*/env_logger/struct.Builder.html#method.parse
[`RUST_LOG`]: https://docs.rs/env_logger/*/env_logger/#enabling-logging

60
src/development_tools/debugging/config_log/log-mod.md Normal file
View file

@ -0,0 +1,60 @@
## Enable log levels per module
[![log-badge]][log] [![env_logger-badge]][env_logger] [![cat-debugging-badge]][cat-debugging]
Creates two modules `foo` and nested `foo::bar` with logging directives
controlled separately with [`RUST_LOG`] environmental variable.
```rust
#[macro_use]
extern crate log;
extern crate env_logger;
mod foo {
mod bar {
pub fn run() {
warn!("[bar] warn");
info!("[bar] info");
debug!("[bar] debug");
}
}
pub fn run() {
warn!("[foo] warn");
info!("[foo] info");
debug!("[foo] debug");
bar::run();
}
}
fn main() {
env_logger::init();
warn!("[root] warn");
info!("[root] info");
debug!("[root] debug");
foo::run();
}
```
[`RUST_LOG`] environment variable controls [`env_logger`][env_logger] output.
Module declarations take comma separated entries formatted like
`path::to::module=log_level`. Run the `test` application as follows:
```bash
RUST_LOG="warn,test::foo=info,test::foo::bar=debug" ./test
```
Sets the default [`log::Level`] to `warn`, module `foo` and module `foo::bar`
to `info` and `debug`.
```bash
WARN:test: [root] warn
WARN:test::foo: [foo] warn
INFO:test::foo: [foo] info
WARN:test::foo::bar: [bar] warn
INFO:test::foo::bar: [bar] info
DEBUG:test::foo::bar: [bar] debug
```
[`log::Level`]: https://docs.rs/log/*/log/enum.Level.html
[`RUST_LOG`]: https://docs.rs/env_logger/*/env_logger/#enabling-logging

55
src/development_tools/debugging/config_log/log-timestamp.md Normal file
View file

@ -0,0 +1,55 @@
## Include timestamp in log messages
[![log-badge]][log] [![env_logger-badge]][env_logger] [![chrono-badge]][chrono] [![cat-debugging-badge]][cat-debugging]
Creates a custom logger configuration with [`Builder`].
Each log entry calls [`Local::now`] to get the current [`DateTime`] in local
timezone and uses [`DateTime::format`] with [`strftime::specifiers`] to format
a timestamp used in the final log.
The example calls [`Builder::format`] to set a closure which formats each
message text with timestamp, [`Record::level`] and body ([`Record::args`]).
```rust
#[macro_use]
extern crate log;
extern crate chrono;
extern crate env_logger;
use std::io::Write;
use chrono::Local;
use env_logger::Builder;
use log::LevelFilter;
fn main() {
Builder::new()
.format(|buf, record| {
writeln!(buf,
"{} [{}] - {}",
Local::now().format("%Y-%m-%dT%H:%M:%S"),
record.level(),
record.args()
)
})
.filter(None, LevelFilter::Info)
.init();
warn!("warn");
info!("info");
debug!("debug");
}
```
stderr output will contain
```
2017-05-22T21:57:06 [WARN] - warn
2017-05-22T21:57:06 [INFO] - info
```
[`DateTime::format`]: https://docs.rs/chrono/*/chrono/struct.DateTime.html#method.format
[`DateTime`]: https://docs.rs/chrono/*/chrono/datetime/struct.DateTime.html
[`Local::now`]: https://docs.rs/chrono/*/chrono/offset/struct.Local.html#method.now
[`Builder`]: https://docs.rs/env_logger/*/env_logger/struct.Builder.html
[`Builder::format`]: https://docs.rs/env_logger/*/env_logger/struct.Builder.html#method.format
[`Record::args`]: https://docs.rs/log/*/log/struct.Record.html#method.args
[`Record::level`]: https://docs.rs/log/*/log/struct.Record.html#method.level
[`strftime::specifiers`]: https://docs.rs/chrono/*/chrono/format/strftime/index.html#specifiers

13
src/development_tools/debugging/log.md Normal file
View file

@ -0,0 +1,13 @@
# Log Messages
{{#include log/log-debug.md}}
{{#include log/log-error.md}}
{{#include log/log-stdout.md}}
{{#include log/log-custom-logger.md}}
{{#include log/log-syslog.md}}
{{#include ../../links.md}}

55
src/development_tools/debugging/log/log-custom-logger.md Normal file
View file

@ -0,0 +1,55 @@
## Log messages with a custom logger
[![log-badge]][log] [![cat-debugging-badge]][cat-debugging]
Implements a custom logger `ConsoleLogger` which prints to stdout.
In order to use the logging macros, `ConsoleLogger` implements
the [`log::Log`] trait and [`log::set_logger`] installs it.
```rust
# #[macro_use]
# extern crate error_chain;
#[macro_use]
extern crate log;
use log::{Record, Level, Metadata, LevelFilter};
static CONSOLE_LOGGER: ConsoleLogger = ConsoleLogger;
struct ConsoleLogger;
impl log::Log for ConsoleLogger {
fn enabled(&self, metadata: &Metadata) -> bool {
metadata.level() <= Level::Info
}
fn log(&self, record: &Record) {
if self.enabled(record.metadata()) {
println!("Rust says: {} - {}", record.level(), record.args());
}
}
fn flush(&self) {}
}
#
# error_chain! {
# foreign_links {
# SetLogger(log::SetLoggerError);
# }
# }
fn run() -> Result<()> {
log::set_logger(&CONSOLE_LOGGER)?;
log::set_max_level(LevelFilter::Info);
info!("hello log");
warn!("warning");
error!("oops");
Ok(())
}
#
# quick_main!(run);
```
[`log::Log`]: https://docs.rs/log/*/log/trait.Log.html
[`log::set_logger`]: https://docs.rs/log/*/log/fn.set_logger.html

42
src/development_tools/debugging/log/log-debug.md Normal file
View file

@ -0,0 +1,42 @@
## Log a debug message to the console
[![log-badge]][log] [![env_logger-badge]][env_logger] [![cat-debugging-badge]][cat-debugging]
The `log` crate provides logging utilities. The `env_logger` crate configures
logging via an environment variable. The [`debug!`] macro works like other
[`std::fmt`] formatted strings.
```rust
#[macro_use]
extern crate log;
extern crate env_logger;
fn execute_query(query: &str) {
debug!("Executing query: {}", query);
}
fn main() {
env_logger::init();
execute_query("DROP TABLE students");
}
```
No output prints when running this code. By default, the
log level is `error`, and any lower levels are dropped.
Set the `RUST_LOG` environment variable to print the message:
```
$ RUST_LOG=debug cargo run
```
Cargo prints debugging information then the
following line at the very end of the output:
```
DEBUG:main: Executing query: DROP TABLE students
```
[`debug!`]: https://docs.rs/log/*/log/macro.debug.html
[`std::fmt`]: https://doc.rust-lang.org/std/fmt/

27
src/development_tools/debugging/log/log-error.md Normal file
View file

@ -0,0 +1,27 @@
## Log an error message to the console
[![log-badge]][log] [![env_logger-badge]][env_logger] [![cat-debugging-badge]][cat-debugging]
Proper error handling considers exceptions exceptional. Here, an error logs
to stderr with `log`'s convenience macro [`error!`].
```rust
#[macro_use]
extern crate log;
extern crate env_logger;
fn execute_query(_query: &str) -> Result<(), &'static str> {
Err("I'm afraid I can't do that")
}
fn main() {
env_logger::init();
let response = execute_query("DROP TABLE students");
if let Err(err) = response {
error!("Failed to execute query: {}", err);
}
}
```
[`error!`]: https://docs.rs/log/*/log/macro.error.html

24
src/development_tools/debugging/log/log-stdout.md Normal file
View file

@ -0,0 +1,24 @@
## Log to stdout instead of stderr
[![log-badge]][log] [![env_logger-badge]][env_logger] [![cat-debugging-badge]][cat-debugging]
Creates a custom logger configuration using the [`Builder::target`] to set the target of the log output to [`Target::Stdout`].
```rust
#[macro_use]
extern crate log;
extern crate env_logger;
use env_logger::{Builder, Target};
fn main() {
Builder::new()
.target(Target::Stdout)
.init();
error!("This error has been printed to Stdout");
}
```
[`Builder::target`]: https://docs.rs/env_logger/*/env_logger/struct.Builder.html#method.target
[`Target::Stdout`]: https://docs.rs/env_logger/*/env_logger/fmt/enum.Target.html

53
src/development_tools/debugging/log/log-syslog.md Normal file
View file

@ -0,0 +1,53 @@
## Log to the Unix syslog
[![log-badge]][log] [![syslog-badge]][syslog] [![cat-debugging-badge]][cat-debugging]
Logs messages to [UNIX syslog]. Initializes logger backend
with [`syslog::init`]. [`syslog::Facility`] records the program submitting
the log entry's classification, [`log::LevelFilter`] denotes allowed log verbosity
and `Option<&str>` holds optional application name.
```rust
# #![allow(unused_imports)]
# #[macro_use]
# extern crate error_chain;
#[macro_use]
extern crate log;
# #[cfg(target_os = "linux")]
extern crate syslog;
# #[cfg(target_os = "linux")]
use syslog::Facility;
#
# #[cfg(target_os = "linux")]
# error_chain! {
# foreign_links {
# SetLogger(syslog::Error);
# }
# }
# #[cfg(target_os = "linux")]
fn run() -> Result<()> {
syslog::init(Facility::LOG_USER,
log::LevelFilter::Debug,
Some("My app name"))?;
debug!("this is a debug {}", "message");
error!("this is an error!");
Ok(())
}
#
# #[cfg(not(target_os = "linux"))]
# error_chain! {}
# #[cfg(not(target_os = "linux"))]
# fn run() -> Result<()> {
# Ok(())
# }
#
# quick_main!(run);
```
[`log::LevelFilter`]: https://docs.rs/log/*/log/enum.LevelFilter.html
[`syslog::Facility`]: https://docs.rs/syslog/*/syslog/enum.Facility.html
[`syslog::init`]: https://docs.rs/syslog/*/syslog/fn.init.html
[UNIX syslog]: https://www.gnu.org/software/libc/manual/html_node/Overview-of-Syslog.html

1
src/development_tools/errors.md Normal file
View file

@ -0,0 +1 @@
# Error Handling

13
src/development_tools/versioning.md Normal file
View file

@ -0,0 +1,13 @@
# Versioning
{{#include versioning/semver-increment.md}}
{{#include versioning/semver-complex.md}}
{{#include versioning/semver-prerelease.md}}
{{#include versioning/semver-latest.md}}
{{#include versioning/semver-command.md}}
{{#include ../links.md}}

57
src/development_tools/versioning/semver-command.md Normal file
View file

@ -0,0 +1,57 @@
## Check external command version for compatibility
[![semver-badge]][semver] [![cat-text-processing-badge]][cat-text-processing] [![cat-os-badge]][cat-os]
Runs `git --version` using [`Command`], then parses the version number into a
[`semver::Version`] using [`Version::parse`]. [`VersionReq::matches`] compares
[`semver::VersionReq`] to the parsed version. The command output resembles
"git version x.y.z".
```rust,no_run
# #[macro_use]
# extern crate error_chain;
extern crate semver;
use std::process::Command;
use semver::{Version, VersionReq};
#
# error_chain! {
# foreign_links {
# Io(std::io::Error);
# Utf8(std::string::FromUtf8Error);
# SemVer(semver::SemVerError);
# SemVerReq(semver::ReqParseError);
# }
# }
fn run() -> Result<()> {
let version_constraint = "> 1.12.0";
let version_test = VersionReq::parse(version_constraint)?;
let output = Command::new("git").arg("--version").output()?;
if !output.status.success() {
bail!("Command executed with failing error code");
}
let stdout = String::from_utf8(output.stdout)?;
let version = stdout.split(" ").last().ok_or_else(|| {
"Invalid command output"
})?;
let parsed_version = Version::parse(version)?;
if !version_test.matches(&parsed_version) {
bail!("Command version lower than minimum supported version (found {}, need {})",
parsed_version, version_constraint);
}
Ok(())
}
#
# quick_main!(run);
```
[`Command`]: https://doc.rust-lang.org/std/process/struct.Command.html
[`semver::Version`]: https://docs.rs/semver/*/semver/struct.Version.html
[`semver::VersionReq`]: https://docs.rs/semver/*/semver/struct.VersionReq.html
[`Version::parse`]: https://docs.rs/semver/*/semver/struct.Version.html#method.parse
[`VersionReq::matches`]: https://docs.rs/semver/*/semver/struct.VersionReq.html#method.matches

55
src/development_tools/versioning/semver-complex.md Normal file
View file

@ -0,0 +1,55 @@
## Parse a complex version string.
[![semver-badge]][semver] [![cat-config-badge]][cat-config]
Constructs a [`semver::Version`] from a complex version string using [`Version::parse`]. The string
contains pre-release and build metadata as defined in the [Semantic Versioning Specification].
Note that, in accordance with the Specification, build metadata is parsed but not considered when
comparing versions. In other words, two versions may be equal even if their build strings differ.
```rust
# #[macro_use]
# extern crate error_chain;
extern crate semver;
use semver::{Identifier, Version};
#
# error_chain! {
# foreign_links {
# SemVer(semver::SemVerError);
# }
# }
fn run() -> Result<()> {
let version_str = "1.0.49-125+g72ee7853";
let parsed_version = Version::parse(version_str)?;
assert_eq!(
parsed_version,
Version {
major: 1,
minor: 0,
patch: 49,
pre: vec![Identifier::Numeric(125)],
build: vec![],
}
);
assert_eq!(
parsed_version.build,
vec![Identifier::AlphaNumeric(String::from("g72ee7853"))]
);
let serialized_version = parsed_version.to_string();
assert_eq!(&serialized_version, version_str);
Ok(())
}
#
# quick_main!(run);
```
[`semver::Version`]: https://docs.rs/semver/*/semver/struct.Version.html
[`Version::parse`]: https://docs.rs/semver/*/semver/struct.Version.html#method.parse
[Semantic Versioning Specification]: http://semver.org/

61
src/development_tools/versioning/semver-increment.md Normal file
View file

@ -0,0 +1,61 @@
## Parse and increment a version string.
[![semver-badge]][semver] [![cat-config-badge]][cat-config]
Constructs a [`semver::Version`] from a string literal using [`Version::parse`],
then increments it by patch, minor, and major version number one by one.
Note that in accordance with the [Semantic Versioning Specification],
incrementing the minor version number resets the patch version number to 0 and
incrementing the major version number resets both the minor and patch version
numbers to 0.
```rust
# #[macro_use]
# extern crate error_chain;
extern crate semver;
use semver::Version;
#
# error_chain! {
# foreign_links {
# SemVer(semver::SemVerError);
# }
# }
fn run() -> Result<()> {
let mut parsed_version = Version::parse("0.2.6")?;
assert_eq!(
parsed_version,
Version {
major: 0,
minor: 2,
patch: 6,
pre: vec![],
build: vec![],
}
);
parsed_version.increment_patch();
assert_eq!(parsed_version.to_string(), "0.2.7");
println!("New patch release: v{}", parsed_version);
parsed_version.increment_minor();
assert_eq!(parsed_version.to_string(), "0.3.0");
println!("New minor release: v{}", parsed_version);
parsed_version.increment_major();
assert_eq!(parsed_version.to_string(), "1.0.0");
println!("New major release: v{}", parsed_version);
Ok(())
}
#
# quick_main!(run);
```
[`semver::Version`]: https://docs.rs/semver/*/semver/struct.Version.html
[`Version::parse`]: https://docs.rs/semver/*/semver/struct.Version.html#method.parse
[Semantic Versioning Specification]: http://semver.org/

66
src/development_tools/versioning/semver-latest.md Normal file
View file

@ -0,0 +1,66 @@
## Find the latest version satisfying given range
[![semver-badge]][semver] [![cat-config-badge]][cat-config]
Given a list of version &strs, finds the latest [`semver::Version`].
[`semver::VersionReq`] filters the list with [`VersionReq::matches`].
Also demonstrates `semver` pre-release preferences.
```rust
# #[macro_use]
# extern crate error_chain;
extern crate semver;
use semver::{Version, VersionReq};
#
# error_chain! {
# foreign_links {
# SemVer(semver::SemVerError);
# SemVerReq(semver::ReqParseError);
# }
# }
fn find_max_matching_version<'a, I>(version_req_str: &str, iterable: I) -> Result<Option<Version>>
where
I: IntoIterator<Item = &'a str>,
{
let vreq = VersionReq::parse(version_req_str)?;
Ok(
iterable
.into_iter()
.filter_map(|s| Version::parse(s).ok())
.filter(|s| vreq.matches(s))
.max(),
)
}
fn run() -> Result<()> {
assert_eq!(
find_max_matching_version("<= 1.0.0", vec!["0.9.0", "1.0.0", "1.0.1"])?,
Some(Version::parse("1.0.0")?)
);
assert_eq!(
find_max_matching_version(
">1.2.3-alpha.3",
vec![
"1.2.3-alpha.3",
"1.2.3-alpha.4",
"1.2.3-alpha.10",
"1.2.3-beta.4",
"3.4.5-alpha.9",
]
)?,
Some(Version::parse("1.2.3-beta.4")?)
);
Ok(())
}
#
# quick_main!(run);
```
[`semver::Version`]: https://docs.rs/semver/*/semver/struct.Version.html
[`semver::VersionReq`]: https://docs.rs/semver/*/semver/struct.VersionReq.html
[`VersionReq::matches`]: https://docs.rs/semver/*/semver/struct.VersionReq.html#method.matches

33
src/development_tools/versioning/semver-prerelease.md Normal file
View file

@ -0,0 +1,33 @@
## Check if given version is pre-release.
[![semver-badge]][semver] [![cat-config-badge]][cat-config]
Given two versions, [`is_prerelease`] asserts that one is pre-release and the other is not.
```rust
# #[macro_use]
# extern crate error_chain;
extern crate semver;
use semver::Version;
#
# error_chain! {
# foreign_links {
# SemVer(semver::SemVerError);
# }
# }
fn run() -> Result<()> {
let version_1 = Version::parse("1.0.0-alpha")?;
let version_2 = Version::parse("1.0.0")?;
assert!(version_1.is_prerelease());
assert!(!version_2.is_prerelease());
Ok(())
}
#
# quick_main!(run);
```
[`is_prerelease`]: https://docs.rs/semver/*/semver/struct.Version.html#method.is_prerelease

852
src/encoding.md
View file

@ -2,8 +2,6 @@
| Recipe | Crates | Categories |
|--------|--------|------------|
| [Serialize and deserialize unstructured JSON][ex-json-value] | [![serde-json-badge]][serde-json] | [![cat-encoding-badge]][cat-encoding] |
| [Deserialize a TOML configuration file][ex-toml-config] | [![toml-badge]][toml] | [![cat-encoding-badge]][cat-encoding] |
| [Percent-encode a string][ex-percent-encode] | [![url-badge]][url] | [![cat-encoding-badge]][cat-encoding] |
| [Encode a string as application/x-www-form-urlencoded][ex-urlencoded] | [![url-badge]][url] | [![cat-encoding-badge]][cat-encoding] |
| [Encode and decode hex][ex-hex-encode-decode] | [![data-encoding-badge]][data-encoding] | [![cat-encoding-badge]][cat-encoding] |
@ -15,837 +13,23 @@
| [Serialize records to CSV][ex-serialize-csv] | [![csv-badge]][csv] | [![cat-encoding-badge]][cat-encoding] |
| [Serialize records to CSV using Serde][ex-csv-serde] | [![csv-badge]][csv] [![serde-badge]][serde] | [![cat-encoding-badge]][cat-encoding] |
| [Transform one column of a CSV file][ex-csv-transform-column] | [![csv-badge]][csv] [![serde-badge]][serde] | [![cat-encoding-badge]][cat-encoding] |
| [Get MIME type from string][ex-mime-from-string] | [![mime-badge]][mime] | [![cat-encoding-badge]][cat-encoding] |
[ex-json-value]: #ex-json-value
<a name="ex-json-value"></a>
## Serialize and deserialize unstructured JSON
[![serde-json-badge]][serde-json] [![cat-encoding-badge]][cat-encoding]
The [serde_json] crate provides a [`serde_json::from_str`] function to parse a `&str` of
JSON into a type of the caller's choice.
Unstructured JSON can be parsed into a universal [`serde_json::Value`] type that
is able to represent any valid JSON data.
The example below shows a `&str` of JSON being parsed and then compared to what
we expect the parsed value to be. The expected value is declared using the
[`json!`] macro.
```rust
# #[macro_use]
# extern crate error_chain;
#[macro_use]
extern crate serde_json;
use serde_json::Value;
#
# error_chain! {
# foreign_links {
# Json(serde_json::Error);
# }
# }
fn run() -> Result<()> {
let j = r#"{
"userid": 103609,
"verified": true,
"access_privileges": [
"user",
"admin"
]
}"#;
let parsed: Value = serde_json::from_str(j)?;
let expected = json!({
"userid": 103609,
"verified": true,
"access_privileges": [
"user",
"admin"
]
});
assert_eq!(parsed, expected);
Ok(())
}
#
# quick_main!(run);
```
[ex-toml-config]: #ex-toml-config
<a name="ex-toml-config"></a>
## Deserialize a TOML configuration file
[![toml-badge]][toml] [![cat-encoding-badge]][cat-encoding]
Parse some TOML into a universal `toml::Value` that is able to represent any
valid TOML data.
```rust
# #[macro_use]
# extern crate error_chain;
extern crate toml;
use toml::Value;
#
# error_chain! {
# foreign_links {
# Toml(toml::de::Error);
# }
# }
fn run() -> Result<()> {
let toml_content = r#"
[package]
name = "your_package"
version = "0.1.0"
authors = ["You! <you@example.org>"]
[dependencies]
serde = "1.0"
"#;
let package_info: Value = toml::from_str(toml_content)?;
assert_eq!(package_info["dependencies"]["serde"].as_str(), Some("1.0"));
assert_eq!(package_info["package"]["name"].as_str(),
Some("your_package"));
Ok(())
}
#
# quick_main!(run);
```
Parse TOML into your own structs using Serde:
[![serde-json-badge]][serde-json] [![toml-badge]][toml] [![cat-encoding-badge]][cat-encoding]
```rust
# #[macro_use]
# extern crate error_chain;
#[macro_use]
extern crate serde_derive;
extern crate toml;
use std::collections::HashMap;
#[derive(Deserialize)]
struct Config {
package: Package,
dependencies: HashMap<String, String>,
}
#[derive(Deserialize)]
struct Package {
name: String,
version: String,
authors: Vec<String>,
}
#
# error_chain! {
# foreign_links {
# Toml(toml::de::Error);
# }
# }
fn run() -> Result<()> {
let toml_content = r#"
[package]
name = "your_package"
version = "0.1.0"
authors = ["You! <you@example.org>"]
[dependencies]
serde = "1.0"
"#;
let package_info: Config = toml::from_str(toml_content)?;
assert_eq!(package_info.package.name, "your_package");
assert_eq!(package_info.package.version, "0.1.0");
assert_eq!(package_info.package.authors, vec!["You! <you@example.org>"]);
assert_eq!(package_info.dependencies["serde"], "1.0");
Ok(())
}
#
# quick_main!(run);
```
[ex-percent-encode]: #ex-percent-encode
<a name="ex-percent-encode"></a>
## Percent-encode a string
[![url-badge]][url] [![cat-encoding-badge]][cat-encoding]
Encode an input string with [percent-encoding] using the [`utf8_percent_encode`]
function from the `url` crate. Then decode using the [`percent_decode`]
function.
```rust
# #[macro_use]
# extern crate error_chain;
extern crate url;
use url::percent_encoding::{utf8_percent_encode, percent_decode, DEFAULT_ENCODE_SET};
#
# error_chain! {
# foreign_links {
# Utf8(std::str::Utf8Error);
# }
# }
fn run() -> Result<()> {
let input = "confident, productive systems programming";
let iter = utf8_percent_encode(input, DEFAULT_ENCODE_SET);
let encoded: String = iter.collect();
assert_eq!(encoded, "confident,%20productive%20systems%20programming");
let iter = percent_decode(encoded.as_bytes());
let decoded = iter.decode_utf8()?;
assert_eq!(decoded, "confident, productive systems programming");
Ok(())
}
#
# quick_main!(run);
```
The encode set defines which bytes (in addition to non-ASCII and controls) need
to be percent-encoded. The choice of this set depends on context. For example,
`?` needs to be encoded in a URL path but not in a query string.
The return value of encoding is an iterator of `&str` slices which can be
collected into a `String`.
[ex-urlencoded]: #ex-urlencoded
<a name="ex-urlencoded"></a>
## Encode a string as application/x-www-form-urlencoded
[![url-badge]][url] [![cat-encoding-badge]][cat-encoding]
Encodes a string into [application/x-www-form-urlencoded] syntax
using the [`form_urlencoded::byte_serialize`] and subsequently
decodes it with [`form_urlencoded::parse`]. Both functions return iterators
that can be collected into a `String`.
```rust
extern crate url;
use url::form_urlencoded::{byte_serialize, parse};
fn main() {
let urlencoded: String = byte_serialize("What is ❤?".as_bytes()).collect();
assert_eq!(urlencoded, "What+is+%E2%9D%A4%3F");
println!("urlencoded:'{}'", urlencoded);
let decoded: String = parse(urlencoded.as_bytes())
.map(|(key, val)| [key, val].concat())
.collect();
assert_eq!(decoded, "What is ❤?");
println!("decoded:'{}'", decoded);
}
```
[ex-hex-encode-decode]: #ex-hex-encode-decode
<a name="ex-hex-encode-decode"></a>
## Encode and decode hex
[![data-encoding-badge]][data-encoding] [![cat-encoding-badge]][cat-encoding]
The [`data_encoding`] crate provides a `HEXUPPER::encode` method which
takes a `&[u8]` and returns a `String` containing the hexadecimal
representation of the data.
Similarly, a `HEXUPPER::decode` method is provided which takes a `&[u8]` and
returns a `Vec<u8>` if the input data is successfully decoded.
[`data_encoding`]: https://github.com/ia0/data-encoding
The example below shows a `&[u8]` of data being converted to its hexadecimal
representation and then being compared to its expected value. The returned
hex `String` is then converted back to its original representation and is
compared to the original value provided.
```rust
# #[macro_use]
# extern crate error_chain;
extern crate data_encoding;
use data_encoding::{HEXUPPER, DecodeError};
#
# error_chain! {
# foreign_links {
# Decode(DecodeError);
# }
# }
fn run() -> Result<()> {
let original = b"The quick brown fox jumps over the lazy dog.";
let expected = "54686520717569636B2062726F776E20666F78206A756D7073206F76\
657220746865206C617A7920646F672E";
let encoded = HEXUPPER.encode(original);
assert_eq!(encoded, expected);
let decoded = HEXUPPER.decode(&encoded.into_bytes())?;
assert_eq!(&decoded[..], &original[..]);
Ok(())
}
#
# quick_main!(run);
```
[ex-base64]: #ex-base64
<a name="ex-base64"></a>
## Encode and decode base64
[![base64-badge]][base64] [![cat-encoding-badge]][cat-encoding]
Encodes byte slice into `base64` String with help of [`encode`]
and subsequently decodes it with [`decode`].
[`decode`]: https://docs.rs/base64/*/base64/fn.decode.html
[`encode`]: https://docs.rs/base64/*/base64/fn.encode.html
```rust
# #[macro_use]
# extern crate error_chain;
extern crate base64;
use std::str;
use base64::{encode, decode};
#
# error_chain! {
# foreign_links {
# Base64(base64::DecodeError);
# Utf8Error(str::Utf8Error);
# }
# }
fn run() -> Result<()> {
let hello = b"hello rustaceans";
let encoded = encode(hello);
let decoded = decode(&encoded)?;
println!("origin: {}", str::from_utf8(hello)?);
println!("base64 encoded: {}", encoded);
println!("back to origin: {}", str::from_utf8(&decoded)?);
Ok(())
}
#
# quick_main!(run);
```
[ex-csv-read]: #ex-csv-read
<a name="ex-csv-read"></a>
## Read CSV records
[![csv-badge]][csv] [![cat-encoding-badge]][cat-encoding]
Reads standard CSV records into [`csv::StringRecord`] — a weakly typed
data representation. It expects to read valid UTF-8 rows. On the
other hand, if invalid UTF-8 data has to be read, then prefer using
[`csv::ByteRecord`], since it makes no assumptions about UTF-8.
```rust
extern crate csv;
# #[macro_use]
# extern crate error_chain;
#
# error_chain! {
# foreign_links {
# Reader(csv::Error);
# }
# }
fn run() -> Result<()> {
let csv = "year,make,model,description
1948,Porsche,356,Luxury sports car
1967,Ford,Mustang fastback 1967,American car";
let mut reader = csv::Reader::from_reader(csv.as_bytes());
for record in reader.records() {
let record = record?;
println!(
"In {}, {} built the {} model. It is a {}.",
&record[0],
&record[1],
&record[2],
&record[3]
);
}
Ok(())
}
#
# quick_main!(run);
```
This is like the previous example, however Serde is used to
deserialize data into strongly type structures. See the
[`csv::Reader::deserialize`] method.
```rust
extern crate csv;
# #[macro_use]
# extern crate error_chain;
#[macro_use]
extern crate serde_derive;
# error_chain! {
# foreign_links {
# Reader(csv::Error);
# }
# }
#
#[derive(Deserialize)]
struct Record {
year: u16,
make: String,
model: String,
description: String,
}
fn run() -> Result<()> {
let csv = "year,make,model,description
1948,Porsche,356,Luxury sports car
1967,Ford,Mustang fastback 1967,American car";
let mut reader = csv::Reader::from_reader(csv.as_bytes());
for record in reader.deserialize() {
let record: Record = record?;
println!(
"In {}, {} built the {} model. It is a {}.",
record.year,
record.make,
record.model,
record.description
);
}
Ok(())
}
#
# quick_main!(run);
```
[ex-csv-delimiter]: #ex-csv-delimiter
<a name="ex-csv-delimiter"></a>
## Read CSV records with different delimiter
[![csv-badge]][csv] [![cat-encoding-badge]][cat-encoding]
Reads CSV records with [`delimiter`] other than ','
[`delimiter`]: https://docs.rs/csv/1.0.0-beta.3/csv/struct.ReaderBuilder.html#method.delimiter
```rust
# #[macro_use]
# extern crate error_chain;
extern crate csv;
#[macro_use]
extern crate serde_derive;
#[derive(Debug, Deserialize)]
struct Record {
name: String,
place: String,
#[serde(deserialize_with = "csv::invalid_option")]
id: Option<u64>,
}
use csv::ReaderBuilder;
#
# error_chain! {
# foreign_links {
# CsvError(csv::Error);
# }
# }
fn run() -> Result<()> {
let data = "name-place-id
Mark-Melbourne-46
Ashley-Zurich-92";
let mut reader = ReaderBuilder::new().delimiter(b'-').from_reader(data.as_bytes());
for result in reader.records() {
println!("{:?}", result?);
}
Ok(())
}
#
# quick_main!(run);
```
[ex-csv-filter]: #ex-csv-filter
<a name="ex-csv-filter"></a>
## Filter CSV records matching a predicate
[![csv-badge]][csv] [![cat-encoding-badge]][cat-encoding]
Returns _only_ the rows from `data` with a field that matches `query`.
```rust
# #[macro_use]
# extern crate error_chain;
extern crate csv;
use std::io;
#
# error_chain!{
# foreign_links {
# Io(std::io::Error);
# CsvError(csv::Error); // or just Seek(csv::Error)
# }
# }
fn run() -> Result<()> {
let query = "CA";
let data = "\
City,State,Population,Latitude,Longitude
Kenai,AK,7610,60.5544444,-151.2583333
Oakman,AL,,33.7133333,-87.3886111
Sandfort,AL,,32.3380556,-85.2233333
West Hollywood,CA,37031,34.0900000,-118.3608333";
let mut rdr = csv::ReaderBuilder::new().from_reader(data.as_bytes());
let mut wtr = csv::Writer::from_writer(io::stdout());
wtr.write_record(rdr.headers()?)?;
for result in rdr.records() {
let record = result?;
if record.iter().any(|field| field == query) {
wtr.write_record(&record)?;
}
}
wtr.flush()?;
Ok(())
}
#
# quick_main!(run);
```
_Disclaimer: this example has been adapted from [the csv crate tutorial](https://docs.rs/csv/*/csv/tutorial/index.html#filter-by-search)_.
[ex-invalid-csv]: #ex-invalid-csv
<a name="ex-invalid-csv"></a>
## Handle invalid CSV data with Serde
[![csv-badge]][csv] [![serde-badge]][serde] [![cat-encoding-badge]][cat-encoding]
CSV files often contain invalid data. For these cases, the csv crate
provides a custom deserializer, [`csv::invalid_option`], which automatically
converts invalid data to None values.
```rust
# #[macro_use]
# extern crate error_chain;
extern crate csv;
#[macro_use]
extern crate serde_derive;
#[derive(Debug, Deserialize)]
struct Record {
name: String,
place: String,
#[serde(deserialize_with = "csv::invalid_option")]
id: Option<u64>,
}
#
# error_chain! {
# foreign_links {
# CsvError(csv::Error);
# }
# }
fn run() -> Result<()> {
let data = "name,place,id
mark,sydney,46.5
ashley,zurich,92
akshat,delhi,37
alisha,colombo,xyz";
let mut rdr = csv::Reader::from_reader(data.as_bytes());
for result in rdr.deserialize() {
let record: Record = result?;
println!("{:?}", record);
}
Ok(())
}
#
# quick_main!(run);
```
[ex-serialize-csv]: #ex-serialize-csv
<a name="ex-serialize-csv"></a>
## Serialize records to CSV
[![csv-badge]][csv] [![cat-encoding-badge]][cat-encoding]
This example shows how to serialize a Rust tuple. [`csv::writer`] supports automatic
serialization from Rust types into CSV records. [`write_record`] is used when writing
a simple record that contains string-like data only, [`serialize`] is used when data
consists of more complex values like numbers, floats or optional values. Since CSV
writer uses internal buffer, always explicitly [`flush`] when done.
```rust
# #[macro_use]
# extern crate error_chain;
extern crate csv;
use std::io;
#
# error_chain! {
# foreign_links {
# CSVError(csv::Error);
# IOError(std::io::Error);
# }
# }
fn run() -> Result<()> {
let mut wtr = csv::Writer::from_writer(io::stdout());
wtr.write_record(&["Name", "Place", "ID"])?;
wtr.serialize(("Mark", "Sydney", 87))?;
wtr.serialize(("Ashley", "Dublin", 32))?;
wtr.serialize(("Akshat", "Delhi", 11))?;
wtr.flush()?;
Ok(())
}
#
# quick_main!(run);
```
[ex-csv-serde]: #ex-csv-serde
<a name="ex-csv-serde"></a>
## Serialize records to CSV using Serde
[![csv-badge]][csv] [![serde-badge]][serde] [![cat-encoding-badge]][cat-encoding]
The following example shows how to serialize custom structs as CSV records using
the [serde] crate.
```rust
# #[macro_use]
# extern crate error_chain;
extern crate csv;
#[macro_use]
extern crate serde_derive;
use std::io;
#
# error_chain! {
# foreign_links {
# IOError(std::io::Error);
# CSVError(csv::Error);
# }
# }
#[derive(Serialize)]
struct Record<'a> {
name: &'a str,
place: &'a str,
id: u64,
}
fn run() -> Result<()> {
let mut wtr = csv::Writer::from_writer(io::stdout());
let rec1 = Record { name: "Mark", place: "Melbourne", id: 56};
let rec2 = Record { name: "Ashley", place: "Sydney", id: 64};
let rec3 = Record { name: "Akshat", place: "Delhi", id: 98};
wtr.serialize(rec1)?;
wtr.serialize(rec2)?;
wtr.serialize(rec3)?;
wtr.flush()?;
Ok(())
}
#
# quick_main!(run);
```
[ex-csv-transform-column]: #ex-csv-transform-column
<a name="ex-csv-transform-column"></a>
## Transform CSV column
[![csv-badge]][csv] [![serde-badge]][serde] [![cat-encoding-badge]][cat-encoding]
Transform a CSV file containing a color name and a hex color into one with a
color name and an rgb color. Utilizes the [csv] crate to read and write the
csv file, and [serde] to deserialize and serialize the rows to and from bytes.
See [`csv::Reader::deserialize`], [`serde::Deserialize`], and [`std::str::FromStr`]
```rust
extern crate csv;
# #[macro_use]
# extern crate error_chain;
#[macro_use]
extern crate serde_derive;
extern crate serde;
use csv::{Reader, Writer};
use serde::{de, Deserialize, Deserializer};
use std::str::FromStr;
#
# error_chain! {
# foreign_links {
# CsvError(csv::Error);
# ParseInt(std::num::ParseIntError);
# CsvInnerError(csv::IntoInnerError<Writer<Vec<u8>>>);
# IO(std::fmt::Error);
# UTF8(std::string::FromUtf8Error);
# }
# }
#[derive(Debug)]
struct HexColor {
red: u8,
green: u8,
blue: u8,
}
#[derive(Debug, Deserialize)]
struct Row {
color_name: String,
color: HexColor,
}
impl FromStr for HexColor {
type Err = Error;
fn from_str(hex_color: &str) -> std::result::Result<Self, Self::Err> {
let trimmed = hex_color.trim_matches('#');
if trimmed.len() != 6 {
Err("Invalid length of hex string".into())
} else {
Ok(HexColor {
red: u8::from_str_radix(&trimmed[..2], 16)?,
green: u8::from_str_radix(&trimmed[2..4], 16)?,
blue: u8::from_str_radix(&trimmed[4..6], 16)?,
})
}
}
}
impl<'de> Deserialize<'de> for HexColor {
fn deserialize<D>(deserializer: D) -> std::result::Result<Self, D::Error>
where
D: Deserializer<'de>,
{
let s = String::deserialize(deserializer)?;
FromStr::from_str(&s).map_err(de::Error::custom)
}
}
fn run() -> Result<()> {
let data = "color_name,color
red,#ff0000
green,#00ff00
blue,#0000FF
periwinkle,#ccccff
magenta,#ff00ff"
.to_owned();
let mut out = Writer::from_writer(vec![]);
let mut reader = Reader::from_reader(data.as_bytes());
for result in reader.deserialize::<Row>() {
let res = result?;
out.serialize((
res.color_name,
res.color.red,
res.color.green,
res.color.blue,
))?;
}
let written = String::from_utf8(out.into_inner()?)?;
assert_eq!(Some("magenta,255,0,255"), written.lines().last());
println!("{}", written);
Ok(())
}
#
# quick_main!(run);
```
[ex-mime-from-string]: #ex-mime-from-string
<a name="ex-mime-from-string"></a>
## Get MIME type from string
[![mime-badge]][mime] [![cat-encoding-badge]][cat-encoding]
The following example shows how to parse a [`MIME`] type from a string using the [mime] crate. You can handle a possible [`FromStrError`] by providing a default [`MIME`] type in an `unwrap_or` clause.
```rust
extern crate mime;
use mime::{Mime, APPLICATION_OCTET_STREAM};
fn main() {
let invalid_mime_type = "i n v a l i d";
let default_mime = invalid_mime_type
.parse::<Mime>()
.unwrap_or(APPLICATION_OCTET_STREAM);
println!(
"MIME for {:?} used default value {:?}",
invalid_mime_type, default_mime
);
let valid_mime_type = "TEXT/PLAIN";
let parsed_mime = valid_mime_type
.parse::<Mime>()
.unwrap_or(APPLICATION_OCTET_STREAM);
println!(
"MIME for {:?} was parsed as {:?}",
valid_mime_type, parsed_mime
);
}
```
| [Serialize and deserialize unstructured JSON][ex-json-value] | [![serde-json-badge]][serde-json] | [![cat-encoding-badge]][cat-encoding] |
| [Deserialize a TOML configuration file][ex-toml-config] | [![toml-badge]][toml] | [![cat-encoding-badge]][cat-encoding] |
| [Read and write integers in little-endian byte order][ex-byteorder-le] | [![byteorder-badge]][byteorder] | [![cat-encoding-badge]][cat-encoding] |
[ex-percent-encode]: encoding/strings.html#percent-encode-a-string
[ex-urlencoded]: encoding/strings.html#encode-a-string-as-applicationx-www-form-urlencoded
[ex-hex-encode-decode]: encoding/strings.html#encode-and-decode-hex
[ex-base64]: encoding/strings.html#encode-and-decode-base64
[ex-csv-read]: encoding/csv.html#read-csv-records
[ex-csv-delimiter]: encoding/csv.html#read-csv-records-with-different-delimiter
[ex-csv-filter]: encoding/csv.html#filter-csv-records-matching-a-predicate
[ex-invalid-csv]: encoding/csv.html#handle-invalid-csv-data-with-serde
[ex-serialize-csv]: encoding/csv.html#serialize-records-to-csv
[ex-csv-serde]: encoding/csv.html#serialize-records-to-csv-using-serde
[ex-csv-transform-column]: encoding/csv.html#transform-csv-column
[ex-json-value]: encoding/complex.html#serialize-and-deserialize-unstructured-json
[ex-toml-config]: encoding/complex.html#deserialize-a-toml-configuration-file
[ex-byteorder-le]: encoding/complex.html#read-and-write-integers-in-little-endian-byte-order
{{#include links.md}}
<!-- API Reference -->
[`csv::ByteRecord`]: https://docs.rs/csv/*/csv/struct.ByteRecord.html
[`csv::invalid_option`]: https://docs.rs/csv/*/csv/fn.invalid_option.html
[`csv::Reader::deserialize`]: https://docs.rs/csv/*/csv/struct.Reader.html#method.deserialize
[`csv::Reader::deserialize`]: https://docs.rs/csv/\*/csv/struct.Reader.html#method.deserialize
[`csv::StringRecord`]: https://docs.rs/csv/*/csv/struct.StringRecord.html
[`csv::Writer`]: https://docs.rs/csv/*/csv/struct.Writer.html
[`flush`]: https://docs.rs/csv/*/csv/struct.Writer.html#method.flush
[`form_urlencoded::byte_serialize`]: https://docs.rs/url/*/url/form_urlencoded/fn.byte_serialize.html
[`form_urlencoded::parse`]: https://docs.rs/url/*/url/form_urlencoded/fn.parse.html
[`FromStrError`]: https://docs.rs/mime/*/mime/struct.FromStrError.html
[`json!`]: https://docs.rs/serde_json/*/serde_json/macro.json.html
[`MIME`]: https://docs.rs/mime/*/mime/struct.Mime.html
[`percent_decode`]: https://docs.rs/percent-encoding/*/percent_encoding/fn.percent_decode.html
[`serde::Deserialize`]: https://docs.rs/serde/\*/serde/trait.Deserialize.html
[`serde_json::from_str`]: https://docs.rs/serde_json/*/serde_json/fn.from_str.html
[`serde_json::Value`]: https://docs.rs/serde_json/*/serde_json/enum.Value.html
[`serialize`]: https://docs.rs/csv/*/csv/struct.Writer.html#method.serialize
[`std::str::FromStr`]: https://doc.rust-lang.org/std/str/trait.FromStr.html
[`utf8_percent_encode`]: https://docs.rs/percent-encoding/*/percent_encoding/fn.utf8_percent_encode.html
[`write_record`]: https://docs.rs/csv/*/csv/struct.Writer.html#method.write_record
<!-- Other Reference -->
[application/x-www-form-urlencoded]: https://url.spec.whatwg.org/#application/x-www-form-urlencoded
[percent-encoding]: https://en.wikipedia.org/wiki/Percent-encoding

9
src/encoding/complex.md Normal file
View file

@ -0,0 +1,9 @@
# Structured Data
{{#include complex/json.md}}
{{#include complex/toml.md}}
{{#include complex/endian-byte.md}}
{{#include ../links.md}}

52
src/encoding/complex/endian-byte.md Normal file
View file

@ -0,0 +1,52 @@
## Read and write integers in little-endian byte order
[![byteorder-badge]][byteorder] [![cat-encoding-badge]][cat-encoding]
`byteorder` can reverse the significant bytes of structured data. This may
be necessary when receiving information over the network, such that bytes
received are from another system.
```rust
# #[macro_use]
# extern crate error_chain;
extern crate byteorder;
use byteorder::{LittleEndian, ReadBytesExt, WriteBytesExt};
#[derive(Default, PartialEq, Debug)]
struct Payload {
kind: u8,
value: u16,
}
#
# error_chain! {
# foreign_links {
# Io(std::io::Error);
# }
# }
fn run() -> Result<()> {
let original_payload = Payload::default();
let encoded_bytes = encode(&original_payload)?;
let decoded_payload = decode(&encoded_bytes)?;
assert_eq!(original_payload, decoded_payload);
Ok(())
}
fn encode(payload: &Payload) -> Result<Vec<u8>> {
let mut bytes = vec![];
bytes.write_u8(payload.kind)?;
bytes.write_u16::<LittleEndian>(payload.value)?;
Ok(bytes)
}
fn decode(mut bytes: &[u8]) -> Result<Payload> {
let payload = Payload {
kind: bytes.read_u8()?,
value: bytes.read_u16::<LittleEndian>()?,
};
Ok(payload)
}
#
# quick_main!(run);
```

59
src/encoding/complex/json.md Normal file
View file

@ -0,0 +1,59 @@
## Serialize and deserialize unstructured JSON
[![serde-json-badge]][serde-json] [![cat-encoding-badge]][cat-encoding]
The [`serde_json`] crate provides a [`from_str`] function to parse a `&str` of
JSON.
Unstructured JSON can be parsed into a universal [`serde_json::Value`] type that
is able to represent any valid JSON data.
The example below shows a `&str` of JSON being parsed. The expected value is declared using the [`json!`] macro.
```rust
# #[macro_use]
# extern crate error_chain;
#[macro_use]
extern crate serde_json;
use serde_json::Value;
#
# error_chain! {
# foreign_links {
# Json(serde_json::Error);
# }
# }
fn run() -> Result<()> {
let j = r#"{
"userid": 103609,
"verified": true,
"access_privileges": [
"user",
"admin"
]
}"#;
let parsed: Value = serde_json::from_str(j)?;
let expected = json!({
"userid": 103609,
"verified": true,
"access_privileges": [
"user",
"admin"
]
});
assert_eq!(parsed, expected);
Ok(())
}
#
# quick_main!(run);
```
[`from_str`]: https://docs.serde.rs/serde_json/fn.from_str.html
[`json!`]: https://docs.serde.rs/serde_json/macro.json.html
[`serde_json`]: https://docs.serde.rs/serde_json/
[`serde_json::Value`]: https://docs.serde.rs/serde_json/enum.Value.html

96
src/encoding/complex/toml.md Normal file
View file

@ -0,0 +1,96 @@
## Deserialize a TOML configuration file
[![toml-badge]][toml] [![cat-encoding-badge]][cat-encoding]
Parse some TOML into a universal `toml::Value` that is able to represent any
valid TOML data.
```rust
# #[macro_use]
# extern crate error_chain;
extern crate toml;
use toml::Value;
#
# error_chain! {
# foreign_links {
# Toml(toml::de::Error);
# }
# }
fn run() -> Result<()> {
let toml_content = r#"
[package]
name = "your_package"
version = "0.1.0"
authors = ["You! <you@example.org>"]
[dependencies]
serde = "1.0"
"#;
let package_info: Value = toml::from_str(toml_content)?;
assert_eq!(package_info["dependencies"]["serde"].as_str(), Some("1.0"));
assert_eq!(package_info["package"]["name"].as_str(),
Some("your_package"));
Ok(())
}
#
# quick_main!(run);
```
Parse TOML into your own structs using [Serde].
```rust
# #[macro_use]
# extern crate error_chain;
#[macro_use]
extern crate serde_derive;
extern crate toml;
use std::collections::HashMap;
#[derive(Deserialize)]
struct Config {
package: Package,
dependencies: HashMap<String, String>,
}
#[derive(Deserialize)]
struct Package {
name: String,
version: String,
authors: Vec<String>,
}
#
# error_chain! {
# foreign_links {
# Toml(toml::de::Error);
# }
# }
fn run() -> Result<()> {
let toml_content = r#"
[package]
name = "your_package"
version = "0.1.0"
authors = ["You! <you@example.org>"]
[dependencies]
serde = "1.0"
"#;
let package_info: Config = toml::from_str(toml_content)?;
assert_eq!(package_info.package.name, "your_package");
assert_eq!(package_info.package.version, "0.1.0");
assert_eq!(package_info.package.authors, vec!["You! <you@example.org>"]);
assert_eq!(package_info.dependencies["serde"], "1.0");
Ok(())
}
#
# quick_main!(run);
```

17
src/encoding/csv.md Normal file
View file

@ -0,0 +1,17 @@
# CSV processing
{{#include csv/read.md}}
{{#include csv/delimiter.md}}
{{#include csv/filter.md}}
{{#include csv/invalid.md}}
{{#include csv/serialize.md}}
{{#include csv/serde-serialize.md}}
{{#include csv/transform.md}}
{{#include ../links.md}}

46
src/encoding/csv/delimiter.md Normal file
View file

@ -0,0 +1,46 @@
## Read CSV records with different delimiter
[![csv-badge]][csv] [![cat-encoding-badge]][cat-encoding]
Reads CSV records with a tab [`delimiter`].
```rust
# #[macro_use]
# extern crate error_chain;
extern crate csv;
#[macro_use]
extern crate serde_derive;
#[derive(Debug, Deserialize)]
struct Record {
name: String,
place: String,
#[serde(deserialize_with = "csv::invalid_option")]
id: Option<u64>,
}
use csv::ReaderBuilder;
#
# error_chain! {
# foreign_links {
# CsvError(csv::Error);
# }
# }
fn run() -> Result<()> {
let data = "name\tplace\tid
Mark\tMelbourne\t46
Ashley\tZurich\t92";
let mut reader = ReaderBuilder::new().delimiter(b'\t').from_reader(data.as_bytes());
for result in reader.records() {
println!("{:?}", result?);
}
Ok(())
}
#
# quick_main!(run);
```
[`delimiter`]: https://docs.rs/csv/1.0.0-beta.3/csv/struct.ReaderBuilder.html#method.delimiter

49
src/encoding/csv/filter.md Normal file
View file

@ -0,0 +1,49 @@
## Filter CSV records matching a predicate
[![csv-badge]][csv] [![cat-encoding-badge]][cat-encoding]
Returns _only_ the rows from `data` with a field that matches `query`.
```rust
# #[macro_use]
# extern crate error_chain;
extern crate csv;
use std::io;
#
# error_chain!{
# foreign_links {
# Io(std::io::Error);
# CsvError(csv::Error);
# }
# }
fn run() -> Result<()> {
let query = "CA";
let data = "\
City,State,Population,Latitude,Longitude
Kenai,AK,7610,60.5544444,-151.2583333
Oakman,AL,,33.7133333,-87.3886111
Sandfort,AL,,32.3380556,-85.2233333
West Hollywood,CA,37031,34.0900000,-118.3608333";
let mut rdr = csv::ReaderBuilder::new().from_reader(data.as_bytes());
let mut wtr = csv::Writer::from_writer(io::stdout());
wtr.write_record(rdr.headers()?)?;
for result in rdr.records() {
let record = result?;
if record.iter().any(|field| field == query) {
wtr.write_record(&record)?;
}
}
wtr.flush()?;
Ok(())
}
#
# quick_main!(run);
```
_Disclaimer: this example has been adapted from [the csv crate tutorial](https://docs.rs/csv/*/csv/tutorial/index.html#filter-by-search)_.

49
src/encoding/csv/invalid.md Normal file
View file

@ -0,0 +1,49 @@
## Handle invalid CSV data with Serde
[![csv-badge]][csv] [![serde-badge]][serde] [![cat-encoding-badge]][cat-encoding]
CSV files often contain invalid data. For these cases, the `csv` crate
provides a custom deserializer, [`csv::invalid_option`], which automatically
converts invalid data to None values.
```rust
# #[macro_use]
# extern crate error_chain;
extern crate csv;
#[macro_use]
extern crate serde_derive;
#[derive(Debug, Deserialize)]
struct Record {
name: String,
place: String,
#[serde(deserialize_with = "csv::invalid_option")]
id: Option<u64>,
}
#
# error_chain! {
# foreign_links {
# CsvError(csv::Error);
# }
# }
fn run() -> Result<()> {
let data = "name,place,id
mark,sydney,46.5
ashley,zurich,92
akshat,delhi,37
alisha,colombo,xyz";
let mut rdr = csv::Reader::from_reader(data.as_bytes());
for result in rdr.deserialize() {
let record: Record = result?;
println!("{:?}", record);
}
Ok(())
}
#
# quick_main!(run);
```
[`csv::invalid_option`]: https://docs.rs/csv/*/csv/fn.invalid_option.html

93
src/encoding/csv/read.md Normal file
View file

@ -0,0 +1,93 @@
## Read CSV records
[![csv-badge]][csv] [![cat-encoding-badge]][cat-encoding]
Reads standard CSV records into [`csv::StringRecord`] — a weakly typed
data representation which expects valid UTF-8 rows. Alternatively,
[`csv::ByteRecord`] makes no assumptions about UTF-8.
```rust
extern crate csv;
# #[macro_use]
# extern crate error_chain;
#
# error_chain! {
# foreign_links {
# Reader(csv::Error);
# }
# }
fn run() -> Result<()> {
let csv = "year,make,model,description
1948,Porsche,356,Luxury sports car
1967,Ford,Mustang fastback 1967,American car";
let mut reader = csv::Reader::from_reader(csv.as_bytes());
for record in reader.records() {
let record = record?;
println!(
"In {}, {} built the {} model. It is a {}.",
&record[0],
&record[1],
&record[2],
&record[3]
);
}
Ok(())
}
#
# quick_main!(run);
```
Serde deserializes data into strongly type structures. See the
[`csv::Reader::deserialize`] method.
```rust
extern crate csv;
# #[macro_use]
# extern crate error_chain;
#[macro_use]
extern crate serde_derive;
# error_chain! {
# foreign_links {
# Reader(csv::Error);
# }
# }
#
#[derive(Deserialize)]
struct Record {
year: u16,
make: String,
model: String,
description: String,
}
fn run() -> Result<()> {
let csv = "year,make,model,description
1948,Porsche,356,Luxury sports car
1967,Ford,Mustang fastback 1967,American car";
let mut reader = csv::Reader::from_reader(csv.as_bytes());
for record in reader.deserialize() {
let record: Record = record?;
println!(
"In {}, {} built the {} model. It is a {}.",
record.year,
record.make,
record.model,
record.description
);
}
Ok(())
}
#
# quick_main!(run);
```
[`csv::ByteRecord`]: https://docs.rs/csv/*/csv/struct.ByteRecord.html
[`csv::Reader::deserialize`]: https://docs.rs/csv/*/csv/struct.Reader.html#method.deserialize
[`csv::StringRecord`]: https://docs.rs/csv/*/csv/struct.StringRecord.html

48
src/encoding/csv/serde-serialize.md Normal file
View file

@ -0,0 +1,48 @@
## Serialize records to CSV using Serde
[![csv-badge]][csv] [![serde-badge]][serde] [![cat-encoding-badge]][cat-encoding]
The following example shows how to serialize custom structs as CSV records using
the [serde] crate.
```rust
# #[macro_use]
# extern crate error_chain;
extern crate csv;
#[macro_use]
extern crate serde_derive;
use std::io;
#
# error_chain! {
# foreign_links {
# IOError(std::io::Error);
# CSVError(csv::Error);
# }
# }
#[derive(Serialize)]
struct Record<'a> {
name: &'a str,
place: &'a str,
id: u64,
}
fn run() -> Result<()> {
let mut wtr = csv::Writer::from_writer(io::stdout());
let rec1 = Record { name: "Mark", place: "Melbourne", id: 56};
let rec2 = Record { name: "Ashley", place: "Sydney", id: 64};
let rec3 = Record { name: "Akshat", place: "Delhi", id: 98};
wtr.serialize(rec1)?;
wtr.serialize(rec2)?;
wtr.serialize(rec3)?;
wtr.flush()?;
Ok(())
}
#
# quick_main!(run);
```

44
src/encoding/csv/serialize.md Normal file
View file

@ -0,0 +1,44 @@
## Serialize records to CSV
[![csv-badge]][csv] [![cat-encoding-badge]][cat-encoding]
This example shows how to serialize a Rust tuple. [`csv::writer`] supports automatic
serialization from Rust types into CSV records. [`write_record`] writes
a simple record containing string data only. Data with more complex values
such as numbers, floats, and options use [`serialize`]. Since CSV
writer uses internal buffer, always explicitly [`flush`] when done.
```rust
# #[macro_use]
# extern crate error_chain;
extern crate csv;
use std::io;
#
# error_chain! {
# foreign_links {
# CSVError(csv::Error);
# IOError(std::io::Error);
# }
# }
fn run() -> Result<()> {
let mut wtr = csv::Writer::from_writer(io::stdout());
wtr.write_record(&["Name", "Place", "ID"])?;
wtr.serialize(("Mark", "Sydney", 87))?;
wtr.serialize(("Ashley", "Dublin", 32))?;
wtr.serialize(("Akshat", "Delhi", 11))?;
wtr.flush()?;
Ok(())
}
#
# quick_main!(run);
```
[`csv::Writer`]: https://docs.rs/csv/*/csv/struct.Writer.html
[`flush`]: https://docs.rs/csv/*/csv/struct.Writer.html#method.flush
[`serialize`]: https://docs.rs/csv/*/csv/struct.Writer.html#method.serialize
[`write_record`]: https://docs.rs/csv/*/csv/struct.Writer.html#method.write_record

104
src/encoding/csv/transform.md Normal file
View file

@ -0,0 +1,104 @@
## Transform CSV column
[![csv-badge]][csv] [![serde-badge]][serde] [![cat-encoding-badge]][cat-encoding]
Transform a CSV file containing a color name and a hex color into one with a
color name and an rgb color. Utilizes the [csv] crate to read and write the
csv file, and [serde] to deserialize and serialize the rows to and from bytes.
See [`csv::Reader::deserialize`], [`serde::Deserialize`], and [`std::str::FromStr`]
```rust
extern crate csv;
# #[macro_use]
# extern crate error_chain;
#[macro_use]
extern crate serde_derive;
extern crate serde;
use csv::{Reader, Writer};
use serde::{de, Deserialize, Deserializer};
use std::str::FromStr;
#
# error_chain! {
# foreign_links {
# CsvError(csv::Error);
# ParseInt(std::num::ParseIntError);
# CsvInnerError(csv::IntoInnerError<Writer<Vec<u8>>>);
# IO(std::fmt::Error);
# UTF8(std::string::FromUtf8Error);
# }
# }
#[derive(Debug)]
struct HexColor {
red: u8,
green: u8,
blue: u8,
}
#[derive(Debug, Deserialize)]
struct Row {
color_name: String,
color: HexColor,
}
impl FromStr for HexColor {
type Err = Error;
fn from_str(hex_color: &str) -> std::result::Result<Self, Self::Err> {
let trimmed = hex_color.trim_matches('#');
if trimmed.len() != 6 {
Err("Invalid length of hex string".into())
} else {
Ok(HexColor {
red: u8::from_str_radix(&trimmed[..2], 16)?,
green: u8::from_str_radix(&trimmed[2..4], 16)?,
blue: u8::from_str_radix(&trimmed[4..6], 16)?,
})
}
}
}
impl<'de> Deserialize<'de> for HexColor {
fn deserialize<D>(deserializer: D) -> std::result::Result<Self, D::Error>
where
D: Deserializer<'de>,
{
let s = String::deserialize(deserializer)?;
FromStr::from_str(&s).map_err(de::Error::custom)
}
}
fn run() -> Result<()> {
let data = "color_name,color
red,#ff0000
green,#00ff00
blue,#0000FF
periwinkle,#ccccff
magenta,#ff00ff"
.to_owned();
let mut out = Writer::from_writer(vec![]);
let mut reader = Reader::from_reader(data.as_bytes());
for result in reader.deserialize::<Row>() {
let res = result?;
out.serialize((
res.color_name,
res.color.red,
res.color.green,
res.color.blue,
))?;
}
let written = String::from_utf8(out.into_inner()?)?;
assert_eq!(Some("magenta,255,0,255"), written.lines().last());
println!("{}", written);
Ok(())
}
#
# quick_main!(run);
```
[`csv::Reader::deserialize`]: https://docs.rs/csv/\*/csv/struct.Reader.html#method.deserialize
[`csv::invalid_option`]: https://docs.rs/csv/*/csv/fn.invalid_option.html
[`serde::Deserialize`]: https://docs.rs/serde/\*/serde/trait.Deserialize.html
[`std::str::FromStr`]: https://doc.rust-lang.org/std/str/trait.FromStr.html

39
src/encoding/string/base64.md Normal file
View file

@ -0,0 +1,39 @@
## Encode and decode base64
[![base64-badge]][base64] [![cat-encoding-badge]][cat-encoding]
Encodes byte slice into `base64` String using [`encode`]
and decodes it with [`decode`].
```rust
# #[macro_use]
# extern crate error_chain;
extern crate base64;
use std::str;
use base64::{encode, decode};
#
# error_chain! {
# foreign_links {
# Base64(base64::DecodeError);
# Utf8Error(str::Utf8Error);
# }
# }
fn run() -> Result<()> {
let hello = b"hello rustaceans";
let encoded = encode(hello);
let decoded = decode(&encoded)?;
println!("origin: {}", str::from_utf8(hello)?);
println!("base64 encoded: {}", encoded);
println!("back to origin: {}", str::from_utf8(&decoded)?);
Ok(())
}
#
# quick_main!(run);
```
[`decode`]: https://docs.rs/base64/*/base64/fn.decode.html
[`encode`]: https://docs.rs/base64/*/base64/fn.encode.html

45
src/encoding/string/hex.md Normal file
View file

@ -0,0 +1,45 @@
## Encode and decode hex
[![data-encoding-badge]][data-encoding] [![cat-encoding-badge]][cat-encoding]
The [`data_encoding`] crate provides a `HEXUPPER::encode` method which
takes a `&[u8]` and returns a `String` containing the hexadecimal
representation of the data.
Similarly, a `HEXUPPER::decode` method is provided which takes a `&[u8]` and
returns a `Vec<u8>` if the input data is successfully decoded.
The example below coverts `&[u8]` data to hexadecimal equivalent. Compares this
value to the expected value.
```rust
# #[macro_use]
# extern crate error_chain;
extern crate data_encoding;
use data_encoding::{HEXUPPER, DecodeError};
#
# error_chain! {
# foreign_links {
# Decode(DecodeError);
# }
# }
fn run() -> Result<()> {
let original = b"The quick brown fox jumps over the lazy dog.";
let expected = "54686520717569636B2062726F776E20666F78206A756D7073206F76\
657220746865206C617A7920646F672E";
let encoded = HEXUPPER.encode(original);
assert_eq!(encoded, expected);
let decoded = HEXUPPER.decode(&encoded.into_bytes())?;
assert_eq!(&decoded[..], &original[..]);
Ok(())
}
#
# quick_main!(run);
```
[`data_encoding`]: https://docs.rs/data-encoding/*/data_encoding/

49
src/encoding/string/percent-encode.md Normal file
View file

@ -0,0 +1,49 @@
## Percent-encode a string
[![url-badge]][url] [![cat-encoding-badge]][cat-encoding]
Encode an input string with [percent-encoding] using the [`utf8_percent_encode`]
function from the `percent-encoding` crate. Then decode using the [`percent_decode`]
function.
```rust
# #[macro_use]
# extern crate error_chain;
extern crate url;
use url::percent_encoding::{utf8_percent_encode, percent_decode, DEFAULT_ENCODE_SET};
#
# error_chain! {
# foreign_links {
# Utf8(std::str::Utf8Error);
# }
# }
fn run() -> Result<()> {
let input = "confident, productive systems programming";
let iter = utf8_percent_encode(input, DEFAULT_ENCODE_SET);
let encoded: String = iter.collect();
assert_eq!(encoded, "confident,%20productive%20systems%20programming");
let iter = percent_decode(encoded.as_bytes());
let decoded = iter.decode_utf8()?;
assert_eq!(decoded, "confident, productive systems programming");
Ok(())
}
#
# quick_main!(run);
```
The encode set defines which bytes (in addition to non-ASCII and controls) need
to be percent-encoded. The choice of this set depends on context. For example,
`url` encodes `?` in a URL path but not in a query string.
The return value of encoding is an iterator of `&str` slices which collect into
a `String`.
[`percent_decode`]: https://docs.rs/percent-encoding/*/percent_encoding/fn.percent_decode.html
[`utf8_percent_encode`]: https://docs.rs/percent-encoding/*/percent_encoding/fn.utf8_percent_encode.html
[percent-encoding]: https://en.wikipedia.org/wiki/Percent-encoding

30
src/encoding/string/url-encode.md Normal file
View file

@ -0,0 +1,30 @@
## Encode a string as application/x-www-form-urlencoded
[![url-badge]][url] [![cat-encoding-badge]][cat-encoding]
Encodes a string into [application/x-www-form-urlencoded] syntax
using the [`form_urlencoded::byte_serialize`] and subsequently
decodes it with [`form_urlencoded::parse`]. Both functions return iterators
that collect into a `String`.
```rust
extern crate url;
use url::form_urlencoded::{byte_serialize, parse};
fn main() {
let urlencoded: String = byte_serialize("What is ❤?".as_bytes()).collect();
assert_eq!(urlencoded, "What+is+%E2%9D%A4%3F");
println!("urlencoded:'{}'", urlencoded);
let decoded: String = parse(urlencoded.as_bytes())
.map(|(key, val)| [key, val].concat())
.collect();
assert_eq!(decoded, "What is ❤?");
println!("decoded:'{}'", decoded);
}
```
[`form_urlencoded::byte_serialize`]: https://docs.rs/url/*/url/form_urlencoded/fn.byte_serialize.html
[`form_urlencoded::parse`]: https://docs.rs/url/*/url/form_urlencoded/fn.parse.html
[application/x-www-form-urlencoded]: https://url.spec.whatwg.org/#application/x-www-form-urlencoded

11
src/encoding/strings.md Normal file
View file

@ -0,0 +1,11 @@
# Character Sets
{{#include string/percent-encode.md}}
{{#include string/url-encode.md}}
{{#include string/hex.md}}
{{#include string/base64.md}}
{{#include ../links.md}}

13
src/errors.md Normal file
View file

@ -0,0 +1,13 @@
# Error Handling
| Recipe | Crates | Categories |
|--------|--------|------------|
| [Handle errors correctly in main][ex-error-chain-simple-error-handling] | [![error-chain-badge]][error-chain] | [![cat-rust-patterns-badge]][cat-rust-patterns] |
| [Avoid discarding errors during error conversions][ex-error-chain-avoid-discarding] | [![error-chain-badge]][error-chain] | [![cat-rust-patterns-badge]][cat-rust-patterns] |
| [Obtain backtrace of complex error scenarios][ex-error-chain-backtrace] | [![error-chain-badge]][error-chain] | [![cat-rust-patterns-badge]][cat-rust-patterns] |
[ex-error-chain-simple-error-handling]: errors/handle.html#handle-errors-correctly-in-main
[ex-error-chain-avoid-discarding]: errors/handle.html#avoid-discarding-errors-during-error-conversions
[ex-error-chain-backtrace]: errors/handle.html#obtain-backtrace-of-complex-error-scenarios
{{#include links.md}}

Some files were not shown because too many files have changed in this diff Show more