Adds parallel map-reduce recipe

See #329

* Adds example in `concurrency.md`
* Adds links to `intro.md`
This commit is contained in:
Jeff Hajewski 2017-10-24 09:20:59 -05:00 committed by Michał Budzyński
parent 57ad48c505
commit 477d82f4b4
2 changed files with 62 additions and 0 deletions

View file

@ -5,6 +5,7 @@
| [Mutate the elements of an array in parallel][ex-rayon-iter-mut] | [![rayon-badge]][rayon] | [![cat-concurrency-badge]][cat-concurrency] | | [Mutate the elements of an array in parallel][ex-rayon-iter-mut] | [![rayon-badge]][rayon] | [![cat-concurrency-badge]][cat-concurrency] |
| [Search items using given predicate in parallel][ex-rayon-parallel-search] | [![rayon-badge]][rayon] | [![cat-concurrency-badge]][cat-concurrency] | | [Search items using given predicate in parallel][ex-rayon-parallel-search] | [![rayon-badge]][rayon] | [![cat-concurrency-badge]][cat-concurrency] |
| [Sort a vector in parallel][ex-rayon-parallel-sort] | [![rayon-badge]][rayon] [![rand-badge]][rand] | [![cat-concurrency-badge]][cat-concurrency] | | [Sort a vector in parallel][ex-rayon-parallel-sort] | [![rayon-badge]][rayon] [![rand-badge]][rand] | [![cat-concurrency-badge]][cat-concurrency] |
| [Map-reduce in parallel][ex-rayon-map-reduce] | [![rayon-badge]][rayon] | [![cat-concurrency-badge]][cat-concurrency] |
| [Generate jpg thumbnails in parallel][ex-rayon-thumbnails] | [![rayon-badge]][rayon] [![glob-badge]][glob] [![image-badge]][image] | [![cat-concurrency-badge]][cat-concurrency][![cat-filesystem-badge]][cat-filesystem] | | [Generate jpg thumbnails in parallel][ex-rayon-thumbnails] | [![rayon-badge]][rayon] [![glob-badge]][glob] [![image-badge]][image] | [![cat-concurrency-badge]][cat-concurrency][![cat-filesystem-badge]][cat-filesystem] |
| [Spawn a short-lived thread][ex-crossbeam-spawn] | [![crossbeam-badge]][crossbeam] | [![cat-concurrency-badge]][cat-concurrency] | | [Spawn a short-lived thread][ex-crossbeam-spawn] | [![crossbeam-badge]][crossbeam] | [![cat-concurrency-badge]][cat-concurrency] |
| [Draw fractal dispatching work to a thread pool][ex-threadpool-fractal] | [![threadpool-badge]][threadpool] [![num-badge]][num] [![num_cpus-badge]][num_cpus] [![image-badge]][image] | [![cat-concurrency-badge]][cat-concurrency][![cat-science-badge]][cat-science][![cat-rendering-badge]][cat-rendering] | | [Draw fractal dispatching work to a thread pool][ex-threadpool-fractal] | [![threadpool-badge]][threadpool] [![num-badge]][num] [![num_cpus-badge]][num_cpus] [![image-badge]][image] | [![cat-concurrency-badge]][cat-concurrency][![cat-science-badge]][cat-science][![cat-rendering-badge]][cat-rendering] |
@ -73,6 +74,61 @@ fn main() {
} }
``` ```
[ex-rayon-map-reduce]: #ex-rayon-map-reduce
<a name="ex-rayon-map-reduce"></a>
## Map-reduce in parallel
[![rayon-badge]][rayon] [![cat-concurrency-badge]][cat-concurrency]
This example uses [`rayon::filter`], [`rayon::map`], and [`rayon::reduce`]
to calculate the coniditional average age of a vector of `Person` objects.
[`rayon::filter`] allows us to (in parallel) conditionally include elements from
a collection that satisfy the given predicate. Similarly, [`rayon::map`] and
[`rayon::reduce`] allow us to transform the filtered elements via a unary
operation and reduce them to a single value via a given binary operation,
respectively. We also show use of [`rayon::sum`], which has the same result as
the reduce operation in this example.
```rust
extern crate rayon;
use rayon::prelude::*;
struct Person {
age: u32,
}
fn main() {
let v: Vec<Person> = vec![
Person { age: 23 },
Person { age: 19 },
Person { age: 42 },
Person { age: 17 },
Person { age: 17 },
Person { age: 31 },
Person { age: 30 },
];
let num_over_30 = v.par_iter().filter(|&x| x.age > 30).count() as f32;
let sum_over_30 = v.par_iter()
.map(|x| x.age)
.filter(|&x| x > 30)
.reduce(|| 0, |x, y| x + y);
let alt_sum_30: u32 = v.par_iter()
.map(|x| x.age)
.filter(|&x| x > 30)
.sum();
let avg_over_30 = sum_over_30 as f32 / num_over_30;
let alt_avg_over_30 = alt_sum_30 as f32/ num_over_30;
assert!((avg_over_30 - alt_avg_over_30).abs() < std::f32::EPSILON);
println!("The average age of people older than 30 is {}", avg_over_30);
}
```
[ex-rayon-parallel-search]: #ex-rayon-parallel-search [ex-rayon-parallel-search]: #ex-rayon-parallel-search
<a name="ex-rayon-parallel-search"></a> <a name="ex-rayon-parallel-search"></a>
## Search items using given predicate in parallel ## Search items using given predicate in parallel
@ -456,7 +512,11 @@ fn run() -> Result<()> {
[`par_iter`]: https://docs.rs/rayon/*/rayon/iter/trait.IntoParallelRefIterator.html#tymethod.par_iter [`par_iter`]: https://docs.rs/rayon/*/rayon/iter/trait.IntoParallelRefIterator.html#tymethod.par_iter
[`par_iter_mut`]: https://docs.rs/rayon/*/rayon/iter/trait.IntoParallelRefMutIterator.html#tymethod.par_iter_mut [`par_iter_mut`]: https://docs.rs/rayon/*/rayon/iter/trait.IntoParallelRefMutIterator.html#tymethod.par_iter_mut
[`par_sort_unstable`]: https://docs.rs/rayon/*/rayon/slice/trait.ParallelSliceMut.html#method.par_sort_unstable [`par_sort_unstable`]: https://docs.rs/rayon/*/rayon/slice/trait.ParallelSliceMut.html#method.par_sort_unstable
[`rayon::filter`]: https://docs.rs/rayon/*/rayon/iter/trait.ParallelIterator.html#method.filter
[`rayon::find_any`]: https://docs.rs/rayon/*/rayon/iter/trait.ParallelIterator.html#method.find_any [`rayon::find_any`]: https://docs.rs/rayon/*/rayon/iter/trait.ParallelIterator.html#method.find_any
[`rayon::map`]: https://docs.rs/rayon/*/rayon/iter/trait.ParallelIterator.html#method.map
[`rayon::reduce`]: https://docs.rs/rayon/*/rayon/iter/trait.ParallelIterator.html#method.reduce
[`rayon::sum`]: https://docs.rs/rayon/*/rayon/iter/trait.ParallelIterator.html#method.sum
[`std::find`]: https://doc.rust-lang.org/std/iter/trait.Iterator.html#method.find [`std::find`]: https://doc.rust-lang.org/std/iter/trait.Iterator.html#method.find
[`Walkdir::new`]: https://docs.rs/walkdir/1.0.7/walkdir/struct.WalkDir.html#method.new [`Walkdir::new`]: https://docs.rs/walkdir/1.0.7/walkdir/struct.WalkDir.html#method.new

View file

@ -69,6 +69,7 @@ community. It needs and welcomes help. For details see
| [Mutate the elements of an array in parallel][ex-rayon-iter-mut] | [![rayon-badge]][rayon] | [![cat-concurrency-badge]][cat-concurrency] | | [Mutate the elements of an array in parallel][ex-rayon-iter-mut] | [![rayon-badge]][rayon] | [![cat-concurrency-badge]][cat-concurrency] |
| [Search items using given predicate in parallel][ex-rayon-parallel-search] | [![rayon-badge]][rayon] | [![cat-concurrency-badge]][cat-concurrency] | | [Search items using given predicate in parallel][ex-rayon-parallel-search] | [![rayon-badge]][rayon] | [![cat-concurrency-badge]][cat-concurrency] |
| [Sort a vector in parallel][ex-rayon-parallel-sort] | [![rayon-badge]][rayon] [![rand-badge]][rand] | [![cat-concurrency-badge]][cat-concurrency] | | [Sort a vector in parallel][ex-rayon-parallel-sort] | [![rayon-badge]][rayon] [![rand-badge]][rand] | [![cat-concurrency-badge]][cat-concurrency] |
| [Map-reduce in parallel][ex-rayon-map-reduce] | [![rayon-badge]][rayon] | [![cat-concurrency-badge]][cat-concurrency] |
| [Generate jpg thumbnails in parallel][ex-rayon-thumbnails] | [![rayon-badge]][rayon] [![glob-badge]][glob] [![image-badge]][image] | [![cat-concurrency-badge]][cat-concurrency][![cat-filesystem-badge]][cat-filesystem] | | [Generate jpg thumbnails in parallel][ex-rayon-thumbnails] | [![rayon-badge]][rayon] [![glob-badge]][glob] [![image-badge]][image] | [![cat-concurrency-badge]][cat-concurrency][![cat-filesystem-badge]][cat-filesystem] |
| [Spawn a short-lived thread][ex-crossbeam-spawn] | [![crossbeam-badge]][crossbeam] | [![cat-concurrency-badge]][cat-concurrency] | | [Spawn a short-lived thread][ex-crossbeam-spawn] | [![crossbeam-badge]][crossbeam] | [![cat-concurrency-badge]][cat-concurrency] |
| [Draw fractal dispatching work to a thread pool][ex-threadpool-fractal] | [![threadpool-badge]][threadpool] [![num-badge]][num] [![num_cpus-badge]][num_cpus] [![image-badge]][image] | [![cat-concurrency-badge]][cat-concurrency][![cat-science-badge]][cat-science][![cat-rendering-badge]][cat-rendering] | | [Draw fractal dispatching work to a thread pool][ex-threadpool-fractal] | [![threadpool-badge]][threadpool] [![num-badge]][num] [![num_cpus-badge]][num_cpus] [![image-badge]][image] | [![cat-concurrency-badge]][cat-concurrency][![cat-science-badge]][cat-science][![cat-rendering-badge]][cat-rendering] |
@ -201,6 +202,7 @@ community. It needs and welcomes help. For details see
[ex-rand-range]: basics.html#ex-rand-range [ex-rand-range]: basics.html#ex-rand-range
[ex-random-port-tcp]: net.html#ex-random-port-tcp [ex-random-port-tcp]: net.html#ex-random-port-tcp
[ex-rayon-iter-mut]: concurrency.html#ex-rayon-iter-mut [ex-rayon-iter-mut]: concurrency.html#ex-rayon-iter-mut
[ex-rayon-map-reduce]: concurrency.html#ex-rayon-map-reduce
[ex-rayon-parallel-search]: concurrency.html#ex-rayon-parallel-search [ex-rayon-parallel-search]: concurrency.html#ex-rayon-parallel-search
[ex-rayon-parallel-sort]: concurrency.html#ex-rayon-parallel-sort [ex-rayon-parallel-sort]: concurrency.html#ex-rayon-parallel-sort
[ex-rayon-thumbnails]: concurrency.html#ex-rayon-thumbnails [ex-rayon-thumbnails]: concurrency.html#ex-rayon-thumbnails