rust/src/librustdoc/lib.rs

391 lines
14 KiB
Rust
Raw Normal View History

2014-01-10 15:05:54 +01:00
// Copyright 2012-2014 The Rust Project Developers. See the COPYRIGHT
// file at the top-level directory of this distribution and at
// http://rust-lang.org/COPYRIGHT.
//
// Licensed under the Apache License, Version 2.0 <LICENSE-APACHE or
// http://www.apache.org/licenses/LICENSE-2.0> or the MIT license
// <LICENSE-MIT or http://opensource.org/licenses/MIT>, at your
// option. This file may not be copied, modified, or distributed
// except according to those terms.
2014-01-10 07:47:26 +01:00
#[crate_id = "rustdoc#0.10-pre"];
#[desc = "rustdoc, the Rust documentation extractor"];
#[license = "MIT/ASL2"];
Add generation of static libraries to rustc This commit implements the support necessary for generating both intermediate and result static rust libraries. This is an implementation of my thoughts in https://mail.mozilla.org/pipermail/rust-dev/2013-November/006686.html. When compiling a library, we still retain the "lib" option, although now there are "rlib", "staticlib", and "dylib" as options for crate_type (and these are stackable). The idea of "lib" is to generate the "compiler default" instead of having too choose (although all are interchangeable). For now I have left the "complier default" to be a dynamic library for size reasons. Of the rust libraries, lib{std,extra,rustuv} will bootstrap with an rlib/dylib pair, but lib{rustc,syntax,rustdoc,rustpkg} will only be built as a dynamic object. I chose this for size reasons, but also because you're probably not going to be embedding the rustc compiler anywhere any time soon. Other than the options outlined above, there are a few defaults/preferences that are now opinionated in the compiler: * If both a .dylib and .rlib are found for a rust library, the compiler will prefer the .rlib variant. This is overridable via the -Z prefer-dynamic option * If generating a "lib", the compiler will generate a dynamic library. This is overridable by explicitly saying what flavor you'd like (rlib, staticlib, dylib). * If no options are passed to the command line, and no crate_type is found in the destination crate, then an executable is generated With this change, you can successfully build a rust program with 0 dynamic dependencies on rust libraries. There is still a dynamic dependency on librustrt, but I plan on removing that in a subsequent commit. This change includes no tests just yet. Our current testing infrastructure/harnesses aren't very amenable to doing flavorful things with linking, so I'm planning on adding a new mode of testing which I believe belongs as a separate commit. Closes #552
2013-11-15 23:03:29 +01:00
#[crate_type = "dylib"];
2014-02-16 06:47:28 +01:00
#[crate_type = "rlib"];
#[allow(deprecated_owned_vector)];
log: Introduce liblog, the old std::logging This commit moves all logging out of the standard library into an external crate. This crate is the new crate which is responsible for all logging macros and logging implementation. A few reasons for this change are: * The crate map has always been a bit of a code smell among rust programs. It has difficulty being loaded on almost all platforms, and it's used almost exclusively for logging and only logging. Removing the crate map is one of the end goals of this movement. * The compiler has a fair bit of special support for logging. It has the __log_level() expression as well as generating a global word per module specifying the log level. This is unfairly favoring the built-in logging system, and is much better done purely in libraries instead of the compiler itself. * Initialization of logging is much easier to do if there is no reliance on a magical crate map being available to set module log levels. * If the logging library can be written outside of the standard library, there's no reason that it shouldn't be. It's likely that we're not going to build the highest quality logging library of all time, so third-party libraries should be able to provide just as high-quality logging systems as the default one provided in the rust distribution. With a migration such as this, the change does not come for free. There are some subtle changes in the behavior of liblog vs the previous logging macros: * The core change of this migration is that there is no longer a physical log-level per module. This concept is still emulated (it is quite useful), but there is now only a global log level, not a local one. This global log level is a reflection of the maximum of all log levels specified. The previously generated logging code looked like: if specified_level <= __module_log_level() { println!(...) } The newly generated code looks like: if specified_level <= ::log::LOG_LEVEL { if ::log::module_enabled(module_path!()) { println!(...) } } Notably, the first layer of checking is still intended to be "super fast" in that it's just a load of a global word and a compare. The second layer of checking is executed to determine if the current module does indeed have logging turned on. This means that if any module has a debug log level turned on, all modules with debug log levels get a little bit slower (they all do more expensive dynamic checks to determine if they're turned on or not). Semantically, this migration brings no change in this respect, but runtime-wise, this will have a perf impact on some code. * A `RUST_LOG=::help` directive will no longer print out a list of all modules that can be logged. This is because the crate map will no longer specify the log levels of all modules, so the list of modules is not known. Additionally, warnings can no longer be provided if a malformed logging directive was supplied. The new "hello world" for logging looks like: #[phase(syntax, link)] extern crate log; fn main() { debug!("Hello, world!"); }
2014-03-09 07:11:44 +01:00
#[feature(globs, struct_variant, managed_boxes, macro_rules, phase)];
extern crate syntax;
extern crate rustc;
extern crate serialize;
extern crate sync;
extern crate getopts;
extern crate collections;
2014-02-14 02:49:11 +01:00
extern crate testing = "test";
2014-02-20 04:08:12 +01:00
extern crate time;
log: Introduce liblog, the old std::logging This commit moves all logging out of the standard library into an external crate. This crate is the new crate which is responsible for all logging macros and logging implementation. A few reasons for this change are: * The crate map has always been a bit of a code smell among rust programs. It has difficulty being loaded on almost all platforms, and it's used almost exclusively for logging and only logging. Removing the crate map is one of the end goals of this movement. * The compiler has a fair bit of special support for logging. It has the __log_level() expression as well as generating a global word per module specifying the log level. This is unfairly favoring the built-in logging system, and is much better done purely in libraries instead of the compiler itself. * Initialization of logging is much easier to do if there is no reliance on a magical crate map being available to set module log levels. * If the logging library can be written outside of the standard library, there's no reason that it shouldn't be. It's likely that we're not going to build the highest quality logging library of all time, so third-party libraries should be able to provide just as high-quality logging systems as the default one provided in the rust distribution. With a migration such as this, the change does not come for free. There are some subtle changes in the behavior of liblog vs the previous logging macros: * The core change of this migration is that there is no longer a physical log-level per module. This concept is still emulated (it is quite useful), but there is now only a global log level, not a local one. This global log level is a reflection of the maximum of all log levels specified. The previously generated logging code looked like: if specified_level <= __module_log_level() { println!(...) } The newly generated code looks like: if specified_level <= ::log::LOG_LEVEL { if ::log::module_enabled(module_path!()) { println!(...) } } Notably, the first layer of checking is still intended to be "super fast" in that it's just a load of a global word and a compare. The second layer of checking is executed to determine if the current module does indeed have logging turned on. This means that if any module has a debug log level turned on, all modules with debug log levels get a little bit slower (they all do more expensive dynamic checks to determine if they're turned on or not). Semantically, this migration brings no change in this respect, but runtime-wise, this will have a perf impact on some code. * A `RUST_LOG=::help` directive will no longer print out a list of all modules that can be logged. This is because the crate map will no longer specify the log levels of all modules, so the list of modules is not known. Additionally, warnings can no longer be provided if a malformed logging directive was supplied. The new "hello world" for logging looks like: #[phase(syntax, link)] extern crate log; fn main() { debug!("Hello, world!"); }
2014-03-09 07:11:44 +01:00
#[phase(syntax, link)]
extern crate log;
2012-04-06 02:30:26 +02:00
use std::cell::RefCell;
use std::local_data;
2013-11-11 07:46:32 +01:00
use std::io;
use std::io::{File, MemWriter};
use std::str;
use serialize::{json, Decodable, Encodable};
pub mod clean;
pub mod core;
pub mod doctree;
pub mod fold;
pub mod html {
pub mod highlight;
pub mod escape;
pub mod format;
pub mod layout;
pub mod markdown;
pub mod render;
pub mod toc;
}
pub mod markdown;
pub mod passes;
pub mod plugins;
pub mod visit_ast;
pub mod test;
pub static SCHEMA_VERSION: &'static str = "0.8.1";
type Pass = (&'static str, // name
fn(clean::Crate) -> plugins::PluginResult, // fn
&'static str); // description
static PASSES: &'static [Pass] = &[
("strip-hidden", passes::strip_hidden,
"strips all doc(hidden) items from the output"),
("unindent-comments", passes::unindent_comments,
"removes excess indentation on comments in order for markdown to like it"),
("collapse-docs", passes::collapse_docs,
"concatenates all document attributes into one document attribute"),
("strip-private", passes::strip_private,
"strips all private items from a crate which cannot be seen externally"),
];
static DEFAULT_PASSES: &'static [&'static str] = &[
2013-09-24 23:09:11 +02:00
"strip-hidden",
"strip-private",
"collapse-docs",
"unindent-comments",
];
local_data_key!(pub ctxtkey: @core::DocContext)
local_data_key!(pub analysiskey: core::CrateAnalysis)
type Output = (clean::Crate, ~[plugins::PluginJson]);
2012-11-19 02:56:50 +01:00
pub fn main() {
std::os::set_exit_status(main_args(std::os::args()));
}
pub fn opts() -> ~[getopts::OptGroup] {
use getopts::*;
~[
optflag("h", "help", "show this help message"),
optflag("", "version", "print rustdoc's version"),
optopt("r", "input-format", "the input type of the specified file",
"[rust|json]"),
optopt("w", "output-format", "the output type to write",
"[html|json]"),
optopt("o", "output", "where to place the output", "PATH"),
optmulti("L", "library-path", "directory to add to crate search path",
"DIR"),
optmulti("", "cfg", "pass a --cfg to rustc", ""),
optmulti("", "plugin-path", "directory to load plugins from", "DIR"),
optmulti("", "passes", "space separated list of passes to also run, a \
value of `list` will print available passes",
"PASSES"),
optmulti("", "plugins", "space separated list of plugins to also load",
"PLUGINS"),
optflag("", "no-defaults", "don't run the default passes"),
optflag("", "test", "run code examples as tests"),
optmulti("", "test-args", "arguments to pass to the test runner",
"ARGS"),
optmulti("", "markdown-css", "CSS files to include via <link> in a rendered Markdown file",
"FILES"),
optmulti("", "markdown-in-header",
"files to include inline in the <head> section of a rendered Markdown file",
"FILES"),
optmulti("", "markdown-before-content",
"files to include inline between <body> and the content of a rendered \
Markdown file",
"FILES"),
optmulti("", "markdown-after-content",
"files to include inline between the content and </body> of a rendered \
Markdown file",
"FILES"),
]
}
pub fn usage(argv0: &str) {
println!("{}", getopts::usage(format!("{} [options] <input>", argv0), opts()));
}
2012-11-19 02:56:50 +01:00
pub fn main_args(args: &[~str]) -> int {
let matches = match getopts::getopts(args.tail(), opts()) {
2014-01-10 15:05:54 +01:00
Ok(m) => m,
Err(err) => {
println!("{}", err.to_err_msg());
2014-01-10 15:05:54 +01:00
return 1;
}
};
if matches.opt_present("h") || matches.opt_present("help") {
usage(args[0]);
return 0;
} else if matches.opt_present("version") {
rustc::version(args[0]);
return 0;
2012-11-19 02:56:50 +01:00
}
if matches.free.len() == 0 {
println!("expected an input file to act on");
return 1;
} if matches.free.len() > 1 {
println!("only one input file may be specified");
return 1;
}
let input = matches.free[0].as_slice();
let libs = matches.opt_strs("L").map(|s| Path::new(s.as_slice()));
let libs = @RefCell::new(libs.move_iter().collect());
let test_args = matches.opt_strs("test-args");
let test_args = test_args.iter().flat_map(|s| s.words()).map(|s| s.to_owned()).to_owned_vec();
let should_test = matches.opt_present("test");
let markdown_input = input.ends_with(".md") || input.ends_with(".markdown");
let output = matches.opt_str("o").map(|s| Path::new(s));
match (should_test, markdown_input) {
(true, true) => return markdown::test(input, libs, test_args),
(true, false) => return test::run(input, libs, test_args),
(false, true) => return markdown::render(input, output.unwrap_or(Path::new("doc")),
&matches),
(false, false) => {}
}
if matches.opt_strs("passes") == ~[~"list"] {
println!("Available passes for running rustdoc:");
for &(name, _, description) in PASSES.iter() {
println!("{:>20s} - {}", name, description);
}
println!("{}", "\nDefault passes for rustdoc:"); // FIXME: #9970
for &name in DEFAULT_PASSES.iter() {
println!("{:>20s}", name);
}
2013-09-25 23:28:20 +02:00
return 0;
}
let (krate, res) = match acquire_input(input, &matches) {
Ok(pair) => pair,
Err(s) => {
println!("input error: {}", s);
return 1;
}
};
info!("going to format");
let started = time::precise_time_ns();
match matches.opt_str("w").as_ref().map(|s| s.as_slice()) {
Some("html") | None => {
match html::render::run(krate, output.unwrap_or(Path::new("doc"))) {
2014-01-30 20:30:21 +01:00
Ok(()) => {}
Err(e) => fail!("failed to generate documentation: {}", e),
}
}
Some("json") => {
match json_output(krate, res, output.unwrap_or(Path::new("doc.json"))) {
2014-01-30 20:30:21 +01:00
Ok(()) => {}
Err(e) => fail!("failed to write json: {}", e),
}
}
Some(s) => {
println!("unknown output format: {}", s);
return 1;
}
}
let ended = time::precise_time_ns();
info!("Took {:.03f}s", (ended as f64 - started as f64) / 1e9f64);
return 0;
}
/// Looks inside the command line arguments to extract the relevant input format
/// and files and then generates the necessary rustdoc output for formatting.
fn acquire_input(input: &str,
matches: &getopts::Matches) -> Result<Output, ~str> {
match matches.opt_str("r").as_ref().map(|s| s.as_slice()) {
Some("rust") => Ok(rust_input(input, matches)),
Some("json") => json_input(input),
Some(s) => Err("unknown input format: " + s),
None => {
if input.ends_with(".json") {
json_input(input)
} else {
Ok(rust_input(input, matches))
}
}
}
}
/// Interprets the input file as a rust source file, passing it through the
/// compiler all the way through the analysis passes. The rustdoc output is then
/// generated from the cleaned AST of the crate.
///
/// This form of input will run all of the plug/cleaning passes
fn rust_input(cratefile: &str, matches: &getopts::Matches) -> Output {
let mut default_passes = !matches.opt_present("no-defaults");
let mut passes = matches.opt_strs("passes");
let mut plugins = matches.opt_strs("plugins");
2012-11-19 02:56:50 +01:00
// First, parse the crate and extract all relevant information.
let libs = matches.opt_strs("L").map(|s| Path::new(s.as_slice()));
let cfgs = matches.opt_strs("cfg");
let cr = Path::new(cratefile);
info!("starting to run rustc");
let (krate, analysis) = std::task::try(proc() {
let cr = cr;
core::run_core(libs.move_iter().collect(), cfgs, &cr)
2014-01-27 04:13:29 +01:00
}).unwrap();
info!("finished with rustc");
local_data::set(analysiskey, analysis);
// Process all of the crate attributes, extracting plugin metadata along
// with the passes which we are supposed to run.
match krate.module.get_ref().doc_list() {
Some(nested) => {
for inner in nested.iter() {
match *inner {
clean::Word(ref x) if "no_default_passes" == *x => {
default_passes = false;
}
clean::NameValue(ref x, ref value) if "passes" == *x => {
for pass in value.words() {
passes.push(pass.to_owned());
}
}
clean::NameValue(ref x, ref value) if "plugins" == *x => {
for p in value.words() {
plugins.push(p.to_owned());
}
}
_ => {}
}
}
}
None => {}
}
if default_passes {
for name in DEFAULT_PASSES.rev_iter() {
passes.unshift(name.to_owned());
}
}
2012-11-19 02:56:50 +01:00
// Load all plugins/passes into a PluginManager
2013-12-31 23:50:38 +01:00
let path = matches.opt_str("plugin-path").unwrap_or(~"/tmp/rustdoc/plugins");
let mut pm = plugins::PluginManager::new(Path::new(path));
for pass in passes.iter() {
let plugin = match PASSES.iter().position(|&(p, _, _)| p == *pass) {
Some(i) => PASSES[i].val1(),
None => {
error!("unknown pass {}, skipping", *pass);
continue
},
};
pm.add_plugin(plugin);
}
info!("loading plugins...");
for pname in plugins.move_iter() {
pm.load_plugin(pname);
}
2012-11-19 02:56:50 +01:00
// Run everything!
info!("Executing passes/plugins");
return pm.run_plugins(krate);
}
2012-11-19 02:56:50 +01:00
/// This input format purely deserializes the json output file. No passes are
/// run over the deserialized output.
fn json_input(input: &str) -> Result<Output, ~str> {
2013-12-31 02:17:45 +01:00
let mut input = match File::open(&Path::new(input)) {
2014-01-30 20:30:21 +01:00
Ok(f) => f,
Err(e) => return Err(format!("couldn't open {}: {}", input, e)),
};
2013-12-31 02:17:45 +01:00
match json::from_reader(&mut input) {
Err(s) => Err(s.to_str()),
Ok(json::Object(obj)) => {
let mut obj = obj;
// Make sure the schema is what we expect
match obj.pop(&~"schema") {
Some(json::String(version)) => {
if version.as_slice() != SCHEMA_VERSION {
return Err(format!("sorry, but I only understand \
version {}", SCHEMA_VERSION))
}
}
2013-11-28 21:22:53 +01:00
Some(..) => return Err(~"malformed json"),
None => return Err(~"expected a schema version"),
}
let krate = match obj.pop(&~"crate") {
Some(json) => {
let mut d = json::Decoder::new(json);
Decodable::decode(&mut d)
}
None => return Err(~"malformed json"),
};
// FIXME: this should read from the "plugins" field, but currently
// Json doesn't implement decodable...
let plugin_output = ~[];
Ok((krate, plugin_output))
}
2013-11-28 21:22:53 +01:00
Ok(..) => Err(~"malformed json input: expected an object at the top"),
}
}
2012-11-19 02:56:50 +01:00
/// Outputs the crate/plugin json as a giant json blob at the specified
/// destination.
fn json_output(krate: clean::Crate, res: ~[plugins::PluginJson],
2014-01-30 20:30:21 +01:00
dst: Path) -> io::IoResult<()> {
// {
// "schema": version,
// "crate": { parsed crate ... },
// "plugins": { output of plugins ... }
// }
let mut json = ~collections::TreeMap::new();
json.insert(~"schema", json::String(SCHEMA_VERSION.to_owned()));
let plugins_json = ~res.move_iter().filter_map(|opt| opt).collect();
// FIXME #8335: yuck, Rust -> str -> JSON round trip! No way to .encode
// straight to the Rust JSON representation.
let crate_json_str = {
2013-11-30 06:26:03 +01:00
let mut w = MemWriter::new();
{
let mut encoder = json::Encoder::new(&mut w as &mut io::Writer);
krate.encode(&mut encoder);
2013-11-30 06:26:03 +01:00
}
str::from_utf8_owned(w.unwrap()).unwrap()
};
let crate_json = match json::from_str(crate_json_str) {
Ok(j) => j,
Err(e) => fail!("Rust generated JSON is invalid: {:?}", e)
};
2012-11-19 02:56:50 +01:00
json.insert(~"crate", crate_json);
json.insert(~"plugins", json::Object(plugins_json));
let mut file = try!(File::create(&dst));
try!(json::Object(json).to_writer(&mut file));
2014-01-30 20:30:21 +01:00
Ok(())
2012-11-19 02:56:50 +01:00
}