An MD5 implementation was originally included in #8097, but, since there are a couple different implementations of that digest algorithm (@alco mentioned his implementation on the mailing list just before I opened that PR), it was suggested that I remove it from that PR and open up a new PR to discuss the different implementations and the best way forward. If anyone wants to discuss a different implementation, feel free to present it here and discuss and compare it to this one. I'll just discuss my implementation and I'll leave it to others to present details of theirs.
This implementation relies on the FixedBuffer struct from cryptoutil.rs for managing the input buffer, just like the Sha1 and Sha2 digest implementations do. I tried manually unrolling the loops in the compression function, but I got slightly worse performance when I did that.
Outside of the #[test]s, I also tested the implementation by generating 1,000 inputs of up to 10MB in size and checking the MD5 digest calculated by this code against the MD5 digest calculated by Java's implementation.
On my computer, I'm getting the following performance:
```
test md5::bench::md5_10 ... bench: 52 ns/iter (+/- 1) = 192 MB/s
test md5::bench::md5_1k ... bench: 2819 ns/iter (+/- 44) = 363 MB/s
test md5::bench::md5_64k ... bench: 178566 ns/iter (+/- 4927) = 367 MB/s
```
Addresses part of #7104
This module adds the ability to generate UUIDs (on all Rust-supported platforms).
I reviewed the existing UUID support in libraries for a range of languages; Go, D, C#, Java and Boost++. The features were all very similar, and this patch essentially covers the union. The implmentation is quite straightforward, and uses the underlying rng support which is assumed to be sufficiently strong for this purpose.
This patch is not complete, however I have put this up for review to gather feedback before finalising. It has tests for most features and documentation for most functions.
Outstanding issues:
* Only generates V4 (Random) UUIDs. Do we want to support the SHA-1 hash based flavour as well?
* Is it worth having the field-based struct public as well as the byte array?
* Formatting the string with '-' between groups not done yet.
* Parsing full string not done as there appears to be no regexp support yet. I can write a simple manual parser for now?
* D has a generator as well. This would be easy to add. However, given the simple interface for creating a new one, and the presence of the macro, is this useful?
* Is it worth having a separate UUID trait and specific implementation? Or should it just have a struct+impl with the same name? Currently it feels weird to have the trait (which can't be named UUID so as to conflict) a separate thing.
* Should the macro be visible at the top level scope?
As this is a first attempt, some code may not be idiomatic. Please comment below...
Thanks for all feedback!
...er
I believe the calls to waitpid are interacting badly with the message passing that goes
on between schedulers and causing us to have very little parallelism in
the test suite. I don't fully understand the sequence of events that causes
the problem here but clearly blocking on waitpid is something that a
well-behaved task should not be doing.
Unfortunately this adds quite a bit of overhead to each test: one thread, two
tasks, three stacks, so there's a tradeoff. The time to execute run-pass on
my 4-core machine goes from ~750s to ~300s.
This should have a pretty good impact on cycle times.
cc @toddaaro
The shift_add_check_overflow and shift_add_check_overflow_tuple functions are
re-written to be more efficient and to make use of the CheckedAdd instrinsic
instead of manually checking for integer overflow.
* The invokation leading_zeros() is removed and replaced with simple integer
comparison. The leading_zeros() method results in a ctpop LLVM instruction
and it may not be efficient on all architectures; integer comparisons,
however, are efficient on just about any architecture.
* The methods lose the ability for the caller to specify a particular shift
value - that functionality wasn't being used and removing it allows for the
code to be simplified.
* Finally, the methods are renamed to add_bytes_to_bits and
add_bytes_to_bits_tuple to reflect their very specific purposes.
This allows the internal implementation details of the TLS keys to be
changed without requiring the update of all the users. (Or, applying
changes that *have* to be applied for the keys to work correctly, e.g.
forcing LLVM to not merge these constants.)
- generate random UUIDs
- convert to and from strings and bytes
- parse common string formats
- implements Zero, Clone, FromStr, ToStr, Eq, TotalEq and Rand
- unit tests and documentation
- parsing error codes and strings
- incorporate feedback from PR review
Retry of PR #8471
Replace the remaining functions marked for issue #8228 with similar functions that are iterator-based.
Change `either::{lefts, rights}` to be iterator-filtering instead of returning a vector.
Replace `map_vec`, `map_vec2`, `iter_vec2` in std::result with three functions:
* `result::collect` gathers `Iterator<Result<V, U>>` to `Result<~[V], U>`
* `result::fold` folds `Iterator<Result<T, E>>` to `Result<V, E>`
* `result::fold_` folds `Iterator<Result<T, E>>` to `Result<(), E>`
This allows the internal implementation details of the TLS keys to be
changed without requiring the update of all the users. (Or, applying
changes that have to be applied for the keys to work correctly, e.g.
forcing LLVM to not merge these constants.)
Fixes#7722
I had a couple of queries:
- Should this return an array or an iterator?
- Should this be a method on iterators or on the rng? I implemented it in RngUtils as it seemed to belong with shuffle().
Rewriting visit.rs to operate on a borrowed `&mut V` where `<V:Visitor>`
r? @nikomatsakis
r? @pcwalton
This is the first in a planned series of incremental pull requests. (There will probably be five pull requests including this one, though they can be combined or split as necessary.)
Part of #7081. (But definitely does *not* complete it, not on its own, and not even after all five parts land; there are still a few loose ends to tie up or trim afterwards.)
The bulk of this change for this particular PR is pnkfelix@3d83010, which has the changes necessary to visit.rs to support everything else that comes later. The other commits are illustrating the standard mechanical transformation that I am applying.
One important point for nearly *all* of these pull requests: I was deliberately *not* trying to be intelligent in the transformation.
* My goal was to minimize code churn, and make the transformation as mechanical as possible.
* For example, I kept the separation between the Visitor struct (corresponding to the earlier vtable of functions that were potentially closed over local state) and the explicitly passed (and clones) visitor Env. I am certain that this is almost always unnecessary, and a later task will be to go through an meld the Env's into the Visitors as appropriate. (My original goal had been to make such melding part of this task; that's why I turned them into a (Env, vtable) tuple way back when. But I digress.)
* Also, my main goal here was to get rid of the record of `@fn`'s as described by the oldvisit.rs API. (This series gets rid of all but one such case; I'm still investigating that.) There is *still* plenty of `@`-boxing left to be removed, I'm sure, and even still some `@fn`'s too; removing all of those is not the goal here; its just to get rid of the encoded protocol of `@fn`'s in the (old)visit API.
To see where things will be going in the future (i.e., to get a sneak-preview of future pull-requests in the series), see:
* https://github.com/pnkfelix/rust/commits/fsk-visitor-vpar-defaults-step1 (that's this one)
* https://github.com/pnkfelix/rust/commits/fsk-visitor-vpar-defaults-step2
* https://github.com/pnkfelix/rust/commits/fsk-visitor-vpar-defaults-step3
* https://github.com/pnkfelix/rust/commits/fsk-visitor-vpar-defaults-step4
* https://github.com/pnkfelix/rust/commits/fsk-visitor-vpar-defaults-step5
* Note that between step 4 and step 5 there is just a single commit, but its a doozy because its the only case where my mechanical transformation did not apply, and thus more serious rewriting was necessary. See commit pnkfelix@da902b2ff3b1e0bee9fc63cf00c449cceea8abf7
If they are on the trait then it is extremely annoying to use them as
generic parameters to a function, e.g. with the iterator param on the trait
itself, if one was to pass an Extendable<int> to a function that filled it
either from a Range or a Map<VecIterator>, one needs to write something
like:
fn foo<E: Extendable<int, Range<int>> +
Extendable<int, Map<&'self int, int, VecIterator<int>>>
(e: &mut E, ...) { ... }
since using a generic, i.e. `foo<E: Extendable<int, I>, I: Iterator<int>>`
means that `foo` takes 2 type parameters, and the caller has to specify them
(which doesn't work anyway, as they'll mismatch with the iterators used in
`foo` itself).
This patch changes it to:
fn foo<E: Extendable<int>>(e: &mut E, ...) { ... }
LLVMConstStringInContext() doesn't need a null-terminated string. It
takes a length instead. Using .to_c_str() here triggers an ICE whenever
the string literal embeds a null, as in "\x00".
.with_c_str() is a replacement for the old .as_c_str(), to avoid
unnecessary boilerplate.
Replace all usages of .to_c_str().with_ref() with .with_c_str().
Previous dicussions about CString suggested that interior nulls should
throw an error. This was never implemented. Add this now, using a
condition (named null_byte) to allow for recovery.
Add method .to_c_str_unchecked() that skips this check.
The calls to waitpid are interacting badly with the message passing that goes
on between schedulers and causing us to have very little parallelism in
the test suite. I don't fully understand the sequence of events that causes
the problem here but clearly blocking on waitpid is something that a
well-behaved task should not be doing.
Unfortunately this adds quite a bit of overhead to each test: one thread, two
tasks, three stacks, so there's a tradeoff. The time to execute run-pass on
my 4-core machine goes from ~750s to ~300s.
