Rollup merge of #44855 - federicomenaquintero:master, r=steveklabnik

Improved docs for CStr, CString, OsStr, OsString

This expands the documentation for those structs and their corresponding traits, per https://github.com/rust-lang/rust/issues/29354

src/libstd/ffi/c_str.rs

@@ -23,19 +23,69 @@ use ptr;
 use slice;
 use str::{self, Utf8Error};
 
-/// A type representing an owned C-compatible string.
+/// A type representing an owned, C-compatible, nul-terminated string with no nul bytes in the
+/// middle.
 ///
-/// This type serves the primary purpose of being able to safely generate a
+/// This type serves the purpose of being able to safely generate a
 /// C-compatible string from a Rust byte slice or vector. An instance of this
 /// type is a static guarantee that the underlying bytes contain no interior 0
-/// bytes and the final byte is 0.
+/// bytes ("nul characters") and that the final byte is 0 ("nul terminator").
 ///
-/// A `CString` is created from either a byte slice or a byte vector. A [`u8`]
+/// `CString` is to [`CStr`] as [`String`] is to [`&str`]: the former
-/// slice can be obtained with the `as_bytes` method. Slices produced from a
+/// in each pair are owned strings; the latter are borrowed
-/// `CString` do *not* contain the trailing nul terminator unless otherwise
+/// references.
-/// specified.
 ///
+/// # Creating a `CString`
+///
+/// A `CString` is created from either a byte slice or a byte vector,
+/// or anything that implements [`Into`]`<`[`Vec`]`<`[`u8`]`>>` (for
+/// example, you can build a `CString` straight out of a [`String`] or
+/// a [`&str`], since both implement that trait).
+///
+/// The [`new`] method will actually check that the provided `&[u8]`
+/// does not have 0 bytes in the middle, and return an error if it
+/// finds one.
+///
+/// # Extracting a raw pointer to the whole C string
+///
+/// `CString` implements a [`as_ptr`] method through the [`Deref`]
+/// trait. This method will give you a `*const c_char` which you can
+/// feed directly to extern functions that expect a nul-terminated
+/// string, like C's `strdup()`.
+///
+/// # Extracting a slice of the whole C string
+///
+/// Alternatively, you can obtain a `&[`[`u8`]`]` slice from a
+/// `CString` with the [`as_bytes`] method. Slices produced in this
+/// way do *not* contain the trailing nul terminator. This is useful
+/// when you will be calling an extern function that takes a `*const
+/// u8` argument which is not necessarily nul-terminated, plus another
+/// argument with the length of the string — like C's `strndup()`.
+/// You can of course get the slice's length with its
+/// [`len`][slice.len] method.
+///
+/// If you need a `&[`[`u8`]`]` slice *with* the nul terminator, you
+/// can use [`as_bytes_with_nul`] instead.
+///
+/// Once you have the kind of slice you need (with or without a nul
+/// terminator), you can call the slice's own
+/// [`as_ptr`][slice.as_ptr] method to get a raw pointer to pass to
+/// extern functions. See the documentation for that function for a
+/// discussion on ensuring the lifetime of the raw pointer.
+///
+/// [`Into`]: ../convert/trait.Into.html
+/// [`Vec`]: ../vec/struct.Vec.html
+/// [`String`]: ../string/struct.String.html
+/// [`&str`]: ../primitive.str.html
 /// [`u8`]: ../primitive.u8.html
+/// [`new`]: #method.new
+/// [`as_bytes`]: #method.as_bytes
+/// [`as_bytes_with_nul`]: #method.as_bytes_with_nul
+/// [`as_ptr`]: #method.as_ptr
+/// [slice.as_ptr]: ../primitive.slice.html#method.as_ptr
+/// [slice.len]: ../primitive.slice.html#method.len
+/// [`Deref`]: ../ops/trait.Deref.html
+/// [`CStr`]: struct.CStr.html
 ///
 /// # Examples
 ///
@@ -48,6 +98,8 @@ use str::{self, Utf8Error};
 ///     fn my_printer(s: *const c_char);
 /// }
 ///
+/// // We are certain that our string doesn't have 0 bytes in the middle,
+/// // so we can .unwrap()
 /// let c_to_print = CString::new("Hello, world!").unwrap();
 /// unsafe {
 ///     my_printer(c_to_print.as_ptr());
@@ -58,7 +110,7 @@ use str::{self, Utf8Error};
 /// # Safety
 ///
 /// `CString` is intended for working with traditional C-style strings
-/// (a sequence of non-null bytes terminated by a single null byte); the
+/// (a sequence of non-nul bytes terminated by a single nul byte); the
 /// primary use case for these kinds of strings is interoperating with C-like
 /// code. Often you will need to transfer ownership to/from that external
 /// code. It is strongly recommended that you thoroughly read through the
@@ -77,17 +129,21 @@ pub struct CString {
 
 /// Representation of a borrowed C string.
 ///
-/// This dynamically sized type is only safely constructed via a borrowed
+/// This type represents a borrowed reference to a nul-terminated
-/// version of an instance of `CString`. This type can be constructed from a raw
+/// array of bytes. It can be constructed safely from a `&[`[`u8`]`]`
-/// C string as well and represents a C string borrowed from another location.
+/// slice, or unsafely from a raw `*const c_char`. It can then be
+/// converted to a Rust [`&str`] by performing UTF-8 validation, or
+/// into an owned [`CString`].
+///
+/// `CStr` is to [`CString`] as [`&str`] is to [`String`]: the former
+/// in each pair are borrowed references; the latter are owned
+/// strings.
 ///
 /// Note that this structure is **not** `repr(C)` and is not recommended to be
-/// placed in the signatures of FFI functions. Instead safe wrappers of FFI
+/// placed in the signatures of FFI functions. Instead, safe wrappers of FFI
 /// functions may leverage the unsafe [`from_ptr`] constructor to provide a safe
 /// interface to other consumers.
 ///
-/// [`from_ptr`]: #method.from_ptr
-///
 /// # Examples
 ///
 /// Inspecting a foreign C string:
@@ -100,7 +156,7 @@ pub struct CString {
 ///
 /// unsafe {
 ///     let slice = CStr::from_ptr(my_string());
-///     println!("string length: {}", slice.to_bytes().len());
+///     println!("string buffer size without nul terminator: {}", slice.to_bytes().len());
 /// }
 /// ```
 ///
@@ -122,8 +178,6 @@ pub struct CString {
 ///
 /// Converting a foreign C string into a Rust [`String`]:
 ///
-/// [`String`]: ../string/struct.String.html
-///
 /// ```no_run
 /// use std::ffi::CStr;
 /// use std::os::raw::c_char;
@@ -138,6 +192,12 @@ pub struct CString {
 ///
 /// println!("string: {}", my_string_safe());
 /// ```
+///
+/// [`u8`]: ../primitive.u8.html
+/// [`&str`]: ../primitive.str.html
+/// [`String`]: ../string/struct.String.html
+/// [`CString`]: struct.CString.html
+/// [`from_ptr`]: #method.from_ptr
 #[derive(Hash)]
 #[stable(feature = "rust1", since = "1.0.0")]
 pub struct CStr {
@@ -148,9 +208,15 @@ pub struct CStr {
     inner: [c_char]
 }
 
-/// An error returned from [`CString::new`] to indicate that a nul byte was found
+/// An error indicating that an interior nul byte was found.
-/// in the vector provided.
 ///
+/// While Rust strings may contain nul bytes in the middle, C strings
+/// can't, as that byte would effectively truncate the string.
+///
+/// This error is created by the [`new`][`CString::new`] method on
+/// [`CString`]. See its documentation for more.
+///
+/// [`CString`]: struct.CString.html
 /// [`CString::new`]: struct.CString.html#method.new
 ///
 /// # Examples
@@ -164,9 +230,16 @@ pub struct CStr {
 #[stable(feature = "rust1", since = "1.0.0")]
 pub struct NulError(usize, Vec<u8>);
 
-/// An error returned from [`CStr::from_bytes_with_nul`] to indicate that a nul
+/// An error indicating that a nul byte was not in the expected position.
-/// byte was found too early in the slice provided or one wasn't found at all.
 ///
+/// The slice used to create a [`CStr`] must have one and only one nul
+/// byte at the end of the slice.
+///
+/// This error is created by the
+/// [`from_bytes_with_nul`][`CStr::from_bytes_with_nul`] method on
+/// [`CStr`]. See its documentation for more.
+///
+/// [`CStr`]: struct.CStr.html
 /// [`CStr::from_bytes_with_nul`]: struct.CStr.html#method.from_bytes_with_nul
 ///
 /// # Examples
@@ -201,9 +274,18 @@ impl FromBytesWithNulError {
     }
 }
 
-/// An error returned from [`CString::into_string`] to indicate that a UTF-8 error
+/// An error indicating invalid UTF-8 when converting a [`CString`] into a [`String`].
-/// was encountered during the conversion.
 ///
+/// `CString` is just a wrapper over a buffer of bytes with a nul
+/// terminator; [`into_string`][`CString::into_string`] performs UTF-8
+/// validation on those bytes and may return this error.
+///
+/// This `struct` is created by the
+/// [`into_string`][`CString::into_string`] method on [`CString`]. See
+/// its documentation for more.
+///
+/// [`String`]: ../string/struct.String.html
+/// [`CString`]: struct.CString.html
 /// [`CString::into_string`]: struct.CString.html#method.into_string
 #[derive(Clone, PartialEq, Eq, Debug)]
 #[stable(feature = "cstring_into", since = "1.7.0")]
@@ -215,8 +297,11 @@ pub struct IntoStringError {
 impl CString {
     /// Creates a new C-compatible string from a container of bytes.
     ///
-    /// This method will consume the provided data and use the underlying bytes
+    /// This function will consume the provided data and use the
-    /// to construct a new string, ensuring that there is a trailing 0 byte.
+    /// underlying bytes to construct a new string, ensuring that
+    /// there is a trailing 0 byte. This trailing 0 byte will be
+    /// appended by this function; the provided data should *not*
+    /// contain any 0 bytes in it.
     ///
     /// # Examples
     ///
@@ -234,9 +319,11 @@ impl CString {
     ///
     /// # Errors
     ///
-    /// This function will return an error if the bytes yielded contain an
+    /// This function will return an error if the supplied bytes contain an
-    /// internal 0 byte. The error returned will contain the bytes as well as
+    /// internal 0 byte. The [`NulError`] returned will contain the bytes as well as
     /// the position of the nul byte.
+    ///
+    /// [`NulError`]: struct.NulError.html
     #[stable(feature = "rust1", since = "1.0.0")]
     pub fn new<T: Into<Vec<u8>>>(t: T) -> Result<CString, NulError> {
         Self::_new(t.into())
@@ -249,8 +336,8 @@ impl CString {
         }
     }
 
-    /// Creates a C-compatible string from a byte vector without checking for
+    /// Creates a C-compatible string by consuming a byte vector,
-    /// interior 0 bytes.
+    /// without checking for interior 0 bytes.
     ///
     /// This method is equivalent to [`new`] except that no runtime assertion
     /// is made that `v` contains no 0 bytes, and it requires an actual
@@ -275,7 +362,7 @@ impl CString {
         CString { inner: v.into_boxed_slice() }
     }
 
-    /// Retakes ownership of a `CString` that was transferred to C.
+    /// Retakes ownership of a `CString` that was transferred to C via [`into_raw`].
     ///
     /// Additionally, the length of the string will be recalculated from the pointer.
     ///
@@ -286,7 +373,14 @@ impl CString {
     /// ownership of a string that was allocated by foreign code) is likely to lead
     /// to undefined behavior or allocator corruption.
     ///
+    /// > **Note:** If you need to borrow a string that was allocated by
+    /// > foreign code, use [`CStr`]. If you need to take ownership of
+    /// > a string that was allocated by foreign code, you will need to
+    /// > make your own provisions for freeing it appropriately, likely
+    /// > with the foreign code's API to do that.
+    ///
     /// [`into_raw`]: #method.into_raw
+    /// [`CStr`]: struct.CStr.html
     ///
     /// # Examples
     ///
@@ -315,11 +409,11 @@ impl CString {
         CString { inner: Box::from_raw(slice as *mut [c_char] as *mut [u8]) }
     }
 
-    /// Transfers ownership of the string to a C caller.
+    /// Consumes the `CString` and transfers ownership of the string to a C caller.
     ///
-    /// The pointer must be returned to Rust and reconstituted using
+    /// The pointer which this function returns must be returned to Rust and reconstituted using
     /// [`from_raw`] to be properly deallocated. Specifically, one
-    /// should *not* use the standard C `free` function to deallocate
+    /// should *not* use the standard C `free()` function to deallocate
     /// this string.
     ///
     /// Failure to call [`from_raw`] will lead to a memory leak.
@@ -351,11 +445,27 @@ impl CString {
         Box::into_raw(self.into_inner()) as *mut c_char
     }
 
-    /// Converts the `CString` into a [`String`] if it contains valid Unicode data.
+    /// Converts the `CString` into a [`String`] if it contains valid UTF-8 data.
     ///
     /// On failure, ownership of the original `CString` is returned.
     ///
     /// [`String`]: ../string/struct.String.html
+    ///
+    /// # Examples
+    ///
+    /// ```
+    /// use std::ffi::CString;
+    ///
+    /// let valid_utf8 = vec![b'f', b'o', b'o'];
+    /// let cstring = CString::new(valid_utf8).unwrap();
+    /// assert_eq!(cstring.into_string().unwrap(), "foo");
+    ///
+    /// let invalid_utf8 = vec![b'f', 0xff, b'o', b'o'];
+    /// let cstring = CString::new(invalid_utf8).unwrap();
+    /// let err = cstring.into_string().err().unwrap();
+    /// assert_eq!(err.utf8_error().valid_up_to(), 1);
+    /// ```
+
     #[stable(feature = "cstring_into", since = "1.7.0")]
     pub fn into_string(self) -> Result<String, IntoStringError> {
         String::from_utf8(self.into_bytes())
@@ -365,10 +475,11 @@ impl CString {
             })
     }
 
-    /// Returns the underlying byte buffer.
+    /// Consumes the `CString` and returns the underlying byte buffer.
     ///
-    /// The returned buffer does **not** contain the trailing nul separator and
+    /// The returned buffer does **not** contain the trailing nul
-    /// it is guaranteed to not have any interior nul bytes.
+    /// terminator, and it is guaranteed to not have any interior nul
+    /// bytes.
     ///
     /// # Examples
     ///
@@ -388,7 +499,7 @@ impl CString {
     }
 
     /// Equivalent to the [`into_bytes`] function except that the returned vector
-    /// includes the trailing nul byte.
+    /// includes the trailing nul terminator.
     ///
     /// [`into_bytes`]: #method.into_bytes
     ///
@@ -408,8 +519,12 @@ impl CString {
 
     /// Returns the contents of this `CString` as a slice of bytes.
     ///
-    /// The returned slice does **not** contain the trailing nul separator and
+    /// The returned slice does **not** contain the trailing nul
-    /// it is guaranteed to not have any interior nul bytes.
+    /// terminator, and it is guaranteed to not have any interior nul
+    /// bytes. If you need the nul terminator, use
+    /// [`as_bytes_with_nul`] instead.
+    ///
+    /// [`as_bytes_with_nul`]: #method.as_bytes_with_nul
     ///
     /// # Examples
     ///
@@ -427,7 +542,7 @@ impl CString {
     }
 
     /// Equivalent to the [`as_bytes`] function except that the returned slice
-    /// includes the trailing nul byte.
+    /// includes the trailing nul terminator.
     ///
     /// [`as_bytes`]: #method.as_bytes
     ///
@@ -598,8 +713,8 @@ impl Default for Box<CStr> {
 }
 
 impl NulError {
-    /// Returns the position of the nul byte in the slice that was provided to
+    /// Returns the position of the nul byte in the slice that caused
-    /// [`CString::new`].
+    /// [`CString::new`] to fail.
     ///
     /// [`CString::new`]: struct.CString.html#method.new
     ///
@@ -711,9 +826,9 @@ impl fmt::Display for IntoStringError {
 }
 
 impl CStr {
-    /// Casts a raw C string to a safe C string wrapper.
+    /// Wraps a raw C string with a safe C string wrapper.
     ///
-    /// This function will cast the provided `ptr` to the `CStr` wrapper which
+    /// This function will wrap the provided `ptr` with a `CStr` wrapper, which
     /// allows inspection and interoperation of non-owned C strings. This method
     /// is unsafe for a number of reasons:
     ///
@@ -753,9 +868,9 @@ impl CStr {
 
     /// Creates a C string wrapper from a byte slice.
     ///
-    /// This function will cast the provided `bytes` to a `CStr` wrapper after
+    /// This function will cast the provided `bytes` to a `CStr`
-    /// ensuring that it is null terminated and does not contain any interior
+    /// wrapper after ensuring that the byte slice is nul-terminated
-    /// nul bytes.
+    /// and does not contain any interior nul bytes.
     ///
     /// # Examples
     ///
@@ -766,7 +881,7 @@ impl CStr {
     /// assert!(cstr.is_ok());
     /// ```
     ///
-    /// Creating a `CStr` without a trailing nul byte is an error:
+    /// Creating a `CStr` without a trailing nul terminator is an error:
     ///
     /// ```
     /// use std::ffi::CStr;
@@ -800,7 +915,7 @@ impl CStr {
     /// Unsafely creates a C string wrapper from a byte slice.
     ///
     /// This function will cast the provided `bytes` to a `CStr` wrapper without
-    /// performing any sanity checks. The provided slice must be null terminated
+    /// performing any sanity checks. The provided slice **must** be nul-terminated
     /// and not contain any interior nul bytes.
     ///
     /// # Examples
@@ -822,7 +937,7 @@ impl CStr {
 
     /// Returns the inner pointer to this C string.
     ///
-    /// The returned pointer will be valid for as long as `self` is and points
+    /// The returned pointer will be valid for as long as `self` is, and points
     /// to a contiguous region of memory terminated with a 0 byte to represent
     /// the end of the string.
     ///
@@ -843,9 +958,9 @@ impl CStr {
     /// ```
     ///
     /// This happens because the pointer returned by `as_ptr` does not carry any
-    /// lifetime information and the string is deallocated immediately after
+    /// lifetime information and the [`CString`] is deallocated immediately after
     /// the `CString::new("Hello").unwrap().as_ptr()` expression is evaluated.
-    /// To fix the problem, bind the string to a local variable:
+    /// To fix the problem, bind the `CString` to a local variable:
     ///
     /// ```no_run
     /// use std::ffi::{CString};
@@ -857,6 +972,11 @@ impl CStr {
     ///     *ptr;
     /// }
     /// ```
+    ///
+    /// This way, the lifetime of the `CString` in `hello` encompasses
+    /// the lifetime of `ptr` and the `unsafe` block.
+    ///
+    /// [`CString`]: struct.CString.html
     #[inline]
     #[stable(feature = "rust1", since = "1.0.0")]
     pub fn as_ptr(&self) -> *const c_char {
@@ -865,11 +985,7 @@ impl CStr {
 
     /// Converts this C string to a byte slice.
     ///
-    /// This function will calculate the length of this string (which normally
+    /// The returned slice will **not** contain the trailing nul terminator that this C
-    /// requires a linear amount of work to be done) and then return the
-    /// resulting slice of `u8` elements.
-    ///
-    /// The returned slice will **not** contain the trailing nul that this C
     /// string has.
     ///
     /// > **Note**: This method is currently implemented as a 0-cost cast, but
@@ -894,7 +1010,7 @@ impl CStr {
     /// Converts this C string to a byte slice containing the trailing 0 byte.
     ///
     /// This function is the equivalent of [`to_bytes`] except that it will retain
-    /// the trailing nul instead of chopping it off.
+    /// the trailing nul terminator instead of chopping it off.
     ///
     /// > **Note**: This method is currently implemented as a 0-cost cast, but
     /// > it is planned to alter its definition in the future to perform the
@@ -918,8 +1034,9 @@ impl CStr {
 
     /// Yields a [`&str`] slice if the `CStr` contains valid UTF-8.
     ///
-    /// This function will calculate the length of this string and check for
+    /// If the contents of the `CStr` are valid UTF-8 data, this
-    /// UTF-8 validity, and then return the [`&str`] if it's valid.
+    /// function will return the corresponding [`&str`] slice. Otherwise,
+    /// it will return an error with details of where UTF-8 validation failed.
     ///
     /// > **Note**: This method is currently implemented to check for validity
     /// > after a 0-cost cast, but it is planned to alter its definition in the
@@ -947,10 +1064,12 @@ impl CStr {
 
     /// Converts a `CStr` into a [`Cow`]`<`[`str`]`>`.
     ///
-    /// This function will calculate the length of this string (which normally
+    /// If the contents of the `CStr` are valid UTF-8 data, this
-    /// requires a linear amount of work to be done) and then return the
+    /// function will return a [`Cow`]`::`[`Borrowed`]`(`[`&str`]`)`
-    /// resulting slice as a [`Cow`]`<`[`str`]`>`, replacing any invalid UTF-8 sequences
+    /// with the the corresponding [`&str`] slice. Otherwise, it will
-    /// with `U+FFFD REPLACEMENT CHARACTER`.
+    /// replace any invalid UTF-8 sequences with `U+FFFD REPLACEMENT
+    /// CHARACTER` and return a [`Cow`]`::`[`Owned`]`(`[`String`]`)`
+    /// with the result.
     ///
     /// > **Note**: This method is currently implemented to check for validity
     /// > after a 0-cost cast, but it is planned to alter its definition in the
@@ -958,7 +1077,9 @@ impl CStr {
     /// > check whenever this method is called.
     ///
     /// [`Cow`]: ../borrow/enum.Cow.html
+    /// [`Borrowed`]: ../borrow/enum.Cow.html#variant.Borrowed
     /// [`str`]: ../primitive.str.html
+    /// [`String`]: ../string/struct.String.html
     ///
     /// # Examples
     ///

src/libstd/ffi/mod.rs

@@ -9,6 +9,157 @@
 // except according to those terms.
 
 //! Utilities related to FFI bindings.
+//!
+//! This module provides utilities to handle data across non-Rust
+//! interfaces, like other programming languages and the underlying
+//! operating system. It is mainly of use for FFI (Foreign Function
+//! Interface) bindings and code that needs to exchange C-like strings
+//! with other languages.
+//!
+//! # Overview
+//!
+//! Rust represents owned strings with the [`String`] type, and
+//! borrowed slices of strings with the [`str`] primitive. Both are
+//! always in UTF-8 encoding, and may contain nul bytes in the middle,
+//! i.e. if you look at the bytes that make up the string, there may
+//! be a `\0` among them. Both `String` and `str` store their length
+//! explicitly; there are no nul terminators at the end of strings
+//! like in C.
+//!
+//! C strings are different from Rust strings:
+//!
+//! * **Encodings** - Rust strings are UTF-8, but C strings may use
+//! other encodings. If you are using a string from C, you should
+//! check its encoding explicitly, rather than just assuming that it
+//! is UTF-8 like you can do in Rust.
+//!
+//! * **Character size** - C strings may use `char` or `wchar_t`-sized
+//! characters; please **note** that C's `char` is different from Rust's.
+//! The C standard leaves the actual sizes of those types open to
+//! interpretation, but defines different APIs for strings made up of
+//! each character type. Rust strings are always UTF-8, so different
+//! Unicode characters will be encoded in a variable number of bytes
+//! each. The Rust type [`char`] represents a '[Unicode scalar
+//! value]', which is similar to, but not the same as, a '[Unicode
+//! code point]'.
+//!
+//! * **Nul terminators and implicit string lengths** - Often, C
+//! strings are nul-terminated, i.e. they have a `\0` character at the
+//! end. The length of a string buffer is not stored, but has to be
+//! calculated; to compute the length of a string, C code must
+//! manually call a function like `strlen()` for `char`-based strings,
+//! or `wcslen()` for `wchar_t`-based ones. Those functions return
+//! the number of characters in the string excluding the nul
+//! terminator, so the buffer length is really `len+1` characters.
+//! Rust strings don't have a nul terminator; their length is always
+//! stored and does not need to be calculated. While in Rust
+//! accessing a string's length is a O(1) operation (becasue the
+//! length is stored); in C it is an O(length) operation because the
+//! length needs to be computed by scanning the string for the nul
+//! terminator.
+//!
+//! * **Internal nul characters** - When C strings have a nul
+//! terminator character, this usually means that they cannot have nul
+//! characters in the middle — a nul character would essentially
+//! truncate the string. Rust strings *can* have nul characters in
+//! the middle, because nul does not have to mark the end of the
+//! string in Rust.
+//!
+//! # Representations of non-Rust strings
+//!
+//! [`CString`] and [`CStr`] are useful when you need to transfer
+//! UTF-8 strings to and from languages with a C ABI, like Python.
+//!
+//! * **From Rust to C:** [`CString`] represents an owned, C-friendly
+//! string: it is nul-terminated, and has no internal nul characters.
+//! Rust code can create a `CString` out of a normal string (provided
+//! that the string doesn't have nul characters in the middle), and
+//! then use a variety of methods to obtain a raw `*mut u8` that can
+//! then be passed as an argument to functions which use the C
+//! conventions for strings.
+//!
+//! * **From C to Rust:** [`CStr`] represents a borrowed C string; it
+//! is what you would use to wrap a raw `*const u8` that you got from
+//! a C function. A `CStr` is guaranteed to be a nul-terminated array
+//! of bytes. Once you have a `CStr`, you can convert it to a Rust
+//! `&str` if it's valid UTF-8, or lossily convert it by adding
+//! replacement characters.
+//!
+//! [`OsString`] and [`OsStr`] are useful when you need to transfer
+//! strings to and from the operating system itself, or when capturing
+//! the output of external commands. Conversions between `OsString`,
+//! `OsStr` and Rust strings work similarly to those for [`CString`]
+//! and [`CStr`].
+//!
+//! * [`OsString`] represents an owned string in whatever
+//! representation the operating system prefers. In the Rust standard
+//! library, various APIs that transfer strings to/from the operating
+//! system use `OsString` instead of plain strings. For example,
+//! [`env::var_os()`] is used to query environment variables; it
+//! returns an `Option<OsString>`. If the environment variable exists
+//! you will get a `Some(os_string)`, which you can *then* try to
+//! convert to a Rust string. This yields a [`Result<>`], so that
+//! your code can detect errors in case the environment variable did
+//! not in fact contain valid Unicode data.
+//!
+//! * [`OsStr`] represents a borrowed reference to a string in a
+//! format that can be passed to the operating system. It can be
+//! converted into an UTF-8 Rust string slice in a similar way to
+//! `OsString`.
+//!
+//! # Conversions
+//!
+//! ## On Unix
+//!
+//! On Unix, [`OsStr`] implements the
+//! `std::os::unix:ffi::`[`OsStrExt`][unix.OsStrExt] trait, which
+//! augments it with two methods, [`from_bytes`] and [`as_bytes`].
+//! These do inexpensive conversions from and to UTF-8 byte slices.
+//!
+//! Additionally, on Unix [`OsString`] implements the
+//! `std::os::unix:ffi::`[`OsStringExt`][unix.OsStringExt] trait,
+//! which provides [`from_vec`] and [`into_vec`] methods that consume
+//! their arguments, and take or produce vectors of [`u8`].
+//!
+//! ## On Windows
+//!
+//! On Windows, [`OsStr`] implements the
+//! `std::os::windows::ffi::`[`OsStrExt`][windows.OsStrExt] trait,
+//! which provides an [`encode_wide`] method. This provides an
+//! iterator that can be [`collect`]ed into a vector of [`u16`].
+//!
+//! Additionally, on Windows [`OsString`] implements the
+//! `std::os::windows:ffi::`[`OsStringExt`][windows.OsStringExt]
+//! trait, which provides a [`from_wide`] method. The result of this
+//! method is an `OsString` which can be round-tripped to a Windows
+//! string losslessly.
+//!
+//! [`String`]: ../string/struct.String.html
+//! [`str`]: ../primitive.str.html
+//! [`char`]: ../primitive.char.html
+//! [`u8`]: ../primitive.u8.html
+//! [`u16`]: ../primitive.u16.html
+//! [Unicode scalar value]: http://www.unicode.org/glossary/#unicode_scalar_value
+//! [Unicode code point]: http://www.unicode.org/glossary/#code_point
+//! [`CString`]: struct.CString.html
+//! [`CStr`]: struct.CStr.html
+//! [`OsString`]: struct.OsString.html
+//! [`OsStr`]: struct.OsStr.html
+//! [`env::set_var()`]: ../env/fn.set_var.html
+//! [`env::var_os()`]: ../env/fn.var_os.html
+//! [`Result<>`]: ../result/enum.Result.html
+//! [unix.OsStringExt]: ../os/unix/ffi/trait.OsStringExt.html
+//! [`from_vec`]: ../os/unix/ffi/trait.OsStringExt.html#tymethod.from_vec
+//! [`into_vec`]: ../os/unix/ffi/trait.OsStringExt.html#tymethod.into_vec
+//! [unix.OsStrExt]: ../os/unix/ffi/trait.OsStrExt.html
+//! [`from_bytes`]: ../os/unix/ffi/trait.OsStrExt.html#tymethod.from_bytes
+//! [`as_bytes`]: ../os/unix/ffi/trait.OsStrExt.html#tymethod.as_bytes
+//! [`OsStrExt`]: ../os/unix/ffi/trait.OsStrExt.html
+//! [windows.OsStrExt]: ../os/windows/ffi/trait.OsStrExt.html
+//! [`encode_wide`]: ../os/windows/ffi/trait.OsStrExt.html#tymethod.encode_wide
+//! [`collect`]: ../iter/trait.Iterator.html#method.collect
+//! [windows.OsStringExt]: ../os/windows/ffi/trait.OsStringExt.html
+//! [`from_wide`]: ../os/windows/ffi/trait.OsStringExt.html#tymethod.from_wide
 
 #![stable(feature = "rust1", since = "1.0.0")]
 

src/libstd/ffi/os_str.rs

@@ -32,18 +32,65 @@ use sys_common::{AsInner, IntoInner, FromInner};
 ///
 /// `OsString` and [`OsStr`] bridge this gap by simultaneously representing Rust
 /// and platform-native string values, and in particular allowing a Rust string
-/// to be converted into an "OS" string with no cost.
+/// to be converted into an "OS" string with no cost if possible.
+///
+/// `OsString` is to [`OsStr`] as [`String`] is to [`&str`]: the former
+/// in each pair are owned strings; the latter are borrowed
+/// references.
+///
+/// # Creating an `OsString`
+///
+/// **From a Rust string**: `OsString` implements
+/// [`From`]`<`[`String`]`>`, so you can use `my_string.`[`from`] to
+/// create an `OsString` from a normal Rust string.
+///
+/// **From slices:** Just like you can start with an empty Rust
+/// [`String`] and then [`push_str`][String.push_str] `&str`
+/// sub-string slices into it, you can create an empty `OsString` with
+/// the [`new`] method and then push string slices into it with the
+/// [`push`] method.
+///
+/// # Extracting a borrowed reference to the whole OS string
+///
+/// You can use the [`as_os_str`] method to get an `&`[`OsStr`] from
+/// an `OsString`; this is effectively a borrowed reference to the
+/// whole string.
+///
+/// # Conversions
+///
+/// See the [module's toplevel documentation about conversions][conversions] for a discussion on
+/// the traits which `OsString` implements for conversions from/to native representations.
 ///
 /// [`OsStr`]: struct.OsStr.html
+/// [`From`]: ../convert/trait.From.html
+/// [`from`]: ../convert/trait.From.html#tymethod.from
+/// [`String`]: ../string/struct.String.html
+/// [`&str`]: ../primitive.str.html
+/// [`u8`]: ../primitive.u8.html
+/// [`u16`]: ../primitive.u16.html
+/// [String.push_str]: ../string/struct.String.html#method.push_str
+/// [`new`]: #method.new
+/// [`push`]: #method.push
+/// [`as_os_str`]: #method.as_os_str
 #[derive(Clone)]
 #[stable(feature = "rust1", since = "1.0.0")]
 pub struct OsString {
     inner: Buf
 }
 
-/// Slices into OS strings (see [`OsString`]).
+/// Borrowed reference to an OS string (see [`OsString`]).
+///
+/// This type represents a borrowed reference to a string in the operating system's preferred
+/// representation.
+///
+/// `OsStr` is to [`OsString`] as [`String`] is to [`&str`]: the former in each pair are borrowed
+/// references; the latter are owned strings.
+///
+/// See the [module's toplevel documentation about conversions][conversions] for a discussion on
+/// the traits which `OsStr` implements for conversions from/to native representations.
 ///
 /// [`OsString`]: struct.OsString.html
+/// [conversions]: index.html#conversions
 #[stable(feature = "rust1", since = "1.0.0")]
 pub struct OsStr {
     inner: Slice

src/libstd/sys/windows/ext/ffi.rs

@@ -9,6 +9,62 @@
 // except according to those terms.
 
 //! Windows-specific extensions to the primitives in the `std::ffi` module.
+//!
+//! # Overview
+//!
+//! For historical reasons, the Windows API uses a form of potentially
+//! ill-formed UTF-16 encoding for strings.  Specifically, the 16-bit
+//! code units in Windows strings may contain [isolated surrogate code
+//! points which are not paired together][ill-formed-utf-16].  The
+//! Unicode standard requires that surrogate code points (those in the
+//! range U+D800 to U+DFFF) always be *paired*, because in the UTF-16
+//! encoding a *surrogate code unit pair* is used to encode a single
+//! character.  For compatibility with code that does not enforce
+//! these pairings, Windows does not enforce them, either.
+//!
+//! While it is not always possible to convert such a string losslessly into
+//! a valid UTF-16 string (or even UTF-8), it is often desirable to be
+//! able to round-trip such a string from and to Windows APIs
+//! losslessly.  For example, some Rust code may be "bridging" some
+//! Windows APIs together, just passing `WCHAR` strings among those
+//! APIs without ever really looking into the strings.
+//!
+//! If Rust code *does* need to look into those strings, it can
+//! convert them to valid UTF-8, possibly lossily, by substituting
+//! invalid sequences with U+FFFD REPLACEMENT CHARACTER, as is
+//! conventionally done in other Rust APIs that deal with string
+//! encodings.
+//!
+//! # `OsStringExt` and `OsStrExt`
+//!
+//! [`OsString`] is the Rust wrapper for owned strings in the
+//! preferred representation of the operating system.  On Windows,
+//! this struct gets augmented with an implementation of the
+//! [`OsStringExt`] trait, which has a [`from_wide`] method.  This
+//! lets you create an [`OsString`] from a `&[u16]` slice; presumably
+//! you get such a slice out of a `WCHAR` Windows API.
+//!
+//! Similarly, [`OsStr`] is the Rust wrapper for borrowed strings from
+//! preferred representation of the operating system.  On Windows, the
+//! [`OsStrExt`] trait provides the [`encode_wide`] method, which
+//! outputs an [`EncodeWide`] iterator.  You can [`collect`] this
+//! iterator, for example, to obtain a `Vec<u16>`; you can later get a
+//! pointer to this vector's contents and feed it to Windows APIs.
+//!
+//! These traits, along with [`OsString`] and [`OsStr`], work in
+//! conjunction so that it is possible to **round-trip** strings from
+//! Windows and back, with no loss of data, even if the strings are
+//! ill-formed UTF-16.
+//!
+//! [ill-formed-utf-16]: https://simonsapin.github.io/wtf-8/#ill-formed-utf-16
+//! [`OsString`]: ../../../ffi/struct.OsString.html
+//! [`OsStr`]: ../../../ffi/struct.OsStr.html
+//! [`OsStringExt`]: trait.OsStringExt.html
+//! [`OsStrExt`]: trait.OsStrExt.html
+//! [`EncodeWide`]: struct.EncodeWide.html
+//! [`from_wide`]: trait.OsStringExt.html#tymethod.from_wide
+//! [`encode_wide`]: trait.OsStrExt.html#tymethod.encode_wide
+//! [`collect`]: ../../../iter/trait.Iterator.html#method.collect
 
 #![stable(feature = "rust1", since = "1.0.0")]