vulkano/DESIGN.md

This document contains the global design decisions made by the vulkano library. It can also be a
good start if you want to contribute to some internal parts of vulkano and don't know how it works.

This document assumes that you're already familiar with Vulkan and does not introduce the various
concepts. However it can still be a good read if you are not so familiar.

If you notice any mistake, feel free to open a PR. If you want to suggest something, feel free to
open a PR as well.

# The three kinds of objects

Vulkano provides wrappers around all objects of the Vulkan API. However these objects are split in
three categories, depending on their access pattern:

- Objects that are not created often and in very small numbers.
- Objects that are typically created at initialization and which are often accessed without mutation
  by performance-critical code.
- Objects that are created, destroyed or modified during performance-critical code, and that
  usually require a synchronization strategy to avoid race conditions.

The first category are objects that are not created often and created in very small numbers:
Instances, Devices, Surfaces, Swapchains. In a typical application each of these objects is only
created once and destroyed when the application exits. Vulkano's API provides a struct that
corresponds to each of these objects, and this struct is typically wrapped in an `Arc`.
Their `new` method in fact returns an `Arc<T>` instead of just a `T` in order to encourage users to
use `Arc`s. You use these objects by cloning them around like you would use objects in a
garbage-collected language such as Java.

The second category are objects like the GraphicsPipeline, ComputePipeline, PipelineLayout,
RenderPass and Framebuffer. They are usually created at initialization and don't perform any
operations themselves, but they describe to the Vulkan implementation operations that we are going
to perform and are thus frequently accessed in order to determine whether the operation that the
vulkano user requested is compliant to what was described. Just like the first category, each of
these objects has a struct that corresponds to them, but in order to make these checks as fast as
possible these structs have a template parameter that describes in a strongly-typed fashion the
operation on the CPU side. This makes it possible to move many checks to compile-time instead of
runtime. More information in another section of this document.

The third category are objects like CommandBuffers, CommandPools, DescriptorSets, DescriptorPools,
Buffers, Images, and memory pools (although not technically a Vulkan object). The way they are
implemented has a huge impact on the performance of the application. Contrary to the first two
categories, each of these objects is represented in vulkano by an unsafe trait (and not by a
struct) that can be freely implemented by the user if they wish. Vulkano provides unsafe structs
such as `UnsafeBuffer`, `UnsafeImage`, etc. which have zero overhead and do not perform any safety
checks, and are the tools used by the safe implementations of the traits. Vulkano also provides
some safe implementations for convenience such as `CpuAccessibleBuffer` or `AttachmentImage`.

# Runtime vs compile-time checks

The second category of objects described above are objects that describe to the Vulkan
implementation an operation that we are going to perform later. For example a `ComputePipeline`
object describes to the Vulkan implementation a compute operation and contains the shader's code
and the list of resources that we are going to bind and that are going to be accessed by the shader.

Since vulkano is a safe library, it needs to check whether the operation the user requests (eg.
executing a compute operation) matches the corresponding `ComputePipeline` (for example, check
that the list of resources passed by the user matches what the compute pipeline expects).
These checks can be expensive. For example when it comes to buffers, vulkano needs to check whether
the layout of the buffers passed by the user is the same as what is expected, by looping through all
the members and following several indirections. If you multiply this by several dozens or hundreds
of operations, it can become very expensive.

In order to reduce the stress caused by these checks, structs such as `ComputePipeline` have a
template parameter which describes the operation. Whenever vulkano performs a check, it queries
the templated object through a trait, and each safety check has its own trait. This means
that we can build strongly-typed objects at compile-time that describe a very precise operation and
whose method implementations are trivial. For example, we can create a `MyComputeOpDesc` type which
implements the `ResourcesListMatch<MyResourcesList>` trait (which was made up for the sake of the
example), and the user will only be able to pass a `MyResourcesList` object for the list of
resources. This moves the check to compile-time and totally eliminates any runtime check. The
compute pipeline is then expressed as `ComputePipeline<MyComputeOpDesc>`.

However this design has a drawback, which is that is can be difficult to explicitly express such a
type. A compute pipeline in the example above could be expressed as
`ComputePipeline<MyComputeOpDesc>`, but in practice these types (like `MyComputeOpDesc`) would be
built by builders and can become extremely long and annoying to put in a struct (just like for
example the type of `(10..).filter(|n| n*2).skip(3).take(5)` can be very long and annoying to put
in a struct). This is especially problematic as it concerns objects that are usually created at
initialization and stay alive for a long time, in other words the kind of objects that you would
put in a struct.

In order to solve this naming problem, all the traits that are used to describe operations must be
boxable so that we can turn `ComputePipeline<Very<Long<And<Complicated, Type>>>>` into
`ComputePipeline<Box<ComputePipelineDesc>>`. This means that we can't use associated types and
templates for any of the trait methods. Ideologically it is a bit annoying to have to restrict
ourselves in what we can do just because the user needs to be able to write out the precise type,
but it's the only pragmatic solution for now.

# Submissions

Any object that can be submitted to a GPU queue (for example a command buffer) implements
the `Submit` trait.

The `Submit` trait provides a function named `build` which returns a `Submission<Self>` object
(where `Self` is the type that implements the `Submit` trait). The `Submission` object must be kept
alive by the user for as long as the GPU hasn't finished executing the submission. Trying to
destroy a `Submission` will block until it is the case. Since the `Submission` holds the object
that was submitted, this object is also kept alive for as long as the GPU hasn't finished executing
it.

For the moment submitting an object always creates a fence, which is how the `Submission` knows
whether the GPU has finished executing it. Eventually this will need to be modified for the sake of
performance.

In order to make the `Submit` trait safer to implement, the method that actually needs to be
implemented is not `build` but `append_submission`. This method uses a API/lifetime trick to
guarantee that the GPU only executes command buffers that outlive the struct that implements
`Submit`.

SAFETY ISSUE HERE HOWEVER: the user can use mem::forget on the Submission and then drop the
objects referenced by it. There are two solutions to this: either store a bunch of Arc<Fence> in
every single object referenced by submissions (eg. pipeline objects), or force the user to use
either Arcs or give ownership of the object. The latter is preferred but not yet implemented.

# Pools

There are three kinds of pools in vulkano: memory pools, descriptor pools, and command pools. Only
the last two are technically Vulkan concepts, but using a memory pool is also a very common
pattern that you are strongly encouraged to embrace when you write a Vulkan application.

These three kinds of pools are each represented in vulkano by a trait. When you use the Vulkan API,
you are expected to create multiple command pools and multiple descriptor pools for maximum
performance. In vulkano however, it is the implementation of the pool trait that is responsible
for managing multiple actual pool objects. In other words a pool in vulkano is just a trait that
provides a method to allocate or free some resource, and the advanced functionality of Vulkan
pools (like resetting a command buffer, resetting a pool, or managing the descriptor pool's
capacity) is handled internally by the implementation of the trait. For example freeing a
command buffer can be implemented by resetting it and reusing it, instead of actually freeing it.

One of the goals of vulkano is to be easy to use by default. Therefore vulkano provides a default
implementation for each of these pools, and the `new` constructors of types that need a pool (ie.
buffers, images, descriptor sets, and command buffers) will use the default implementation. It is
possible for the user to use an alternative implementation of a pool by using an alternative
constructor, but the default implementations should be good for most usages. This is similar to
memory allocators in languages such as C++ and Rust, in the sense that some users want to be able
to use a custom allocator but most of the time it's not worth bothering with that.

# Command buffers

Command buffer objects belong to the last category of objects that were described above. They are
represented by an unsafe trait and can be implemented manually by the user if they wish.

However this poses a practical problem, which is that creating a command buffer in a safe way
is really complicated. There are tons of commands to implement, and each command has a ton of
safety requirements. If a user wants to create a custom command buffer type, it is just not an
option to ask them to reimplement these safety checks themselves.

The reason why users may want to create their own command buffer types is to implement
synchronization themselves. Vulkano's default implementation (which is `AutobarriersCommandBuffer`)
will automatically place pipeline barriers in order to handle cache flushes and image layout
transitions and avoid data races, but this automatic computation can be seen as expensive.

In order to make it possible to customize the synchronization story of command buffers, vulkano has
split the command buffer building process in two steps. First the user builds a list of commands
through an iterator-like API (and vulkano will check their validity), and then they are turned into
a command buffer through a trait. This means that the user can customize the synchronization
strategy (by customizing the second step) while still using the same command-building process
(the first step). Commands are not opinionated towards one strategy or another. The
command-building code is totally isolated from the synchronization strategy and only checks
whether the commands themselves are valid.

The fact that all the commands are added at once can be a little surprising for a user coming from
Vulkan. Vulkano's API looks very similar to Vulkan's API, but there is a major difference: in
Vulkan the cost of creating a command buffer is distributed between each function call, but in
vulkano it is done all at once. For example creating a command buffer with 6 commands with Vulkan
requires 8 function calls that take say 5µs each, while creating the same command buffer with
vulkano requires 8 function calls, but the first 7 are almost free and the last one takes 40µs.
After some thinking, it was considered to not be a problem.

Creating a list of commands with an iterator-like API has the problem that the type of the list of
commands changes every time you add a new command to the list
(just like for example `let iterator = iterator.skip(1)` changes the type of `iterator`). This is
a problem in situations where we don't know at compile-time the number of commands that we are
going to add. In order to solve this, it is required that the `CommandsList` trait be boxable,
so that the user can use a `Box<CommandsList>`. This is unfortunately not optimal as you will need
a memory allocation for each command that is added to the list. The situation here could still be
improved.

# The auto-barriers builder

As explained above, the default implementation of a command buffer provided by vulkano
automatically places pipeline barriers to avoid issues such as caches not being flushed, commands
being executed simultaneously when they shouldn't, or images having the wrong layout.

This is not an easy job, because Vulkan allows lots of weird access patterns that we want to make
available in vulkano. You can for example create a buffer object split into multiple sub-buffer
objects, or make some images and buffers share the same memory.

In order to make it possible to handle everything properly, the `Buffer` and `Image` traits need to
help us with the `conflicts` methods. Each buffer and image can be queried to know whether it
potentially uses the same memory as any other buffer or image. When two resources conflict, this
means that you can't write to one and read from the other one simultaneously or write to both
simultaneously.

But we don't want to check every single combination of buffer and image every time to check whether
they conflict. So in order to improve performance, buffers and images also need to provide a key
that identifies them. Two resources that can potentially conflict must always return the same key.
The regular `conflict` functions are still necessary to handle the situation where buffers or
images accidentally return the same key but don't actually conflict.

This conflict system is also used to make sure that the attachments of a framebuffer don't conflict
with each other or that the resources in a descriptor set don't conflict with each other (both
situations are forbidden).

# Image layouts

Tracking image layouts can be tedious. Vulkano uses a simple solution, which is that images must
always be in a specific layout at the beginning and the end of a command buffer. If a transition
is performed during a command buffer, the image must be transitioned back before the end of the
command buffer. The layout in question is queried with a method on the `Image` trait.

For example an `AttachmentImage` must always be in the `ColorAttachmentOptimal` layout for color
attachment, and the `DepthStencilAttachmentOptimal` layout for depth-stencil attachments. If any
command switches the image to another layout, then it will need to be switched back before the end
of the command buffer.

This system works very nicely in practice, and unnecessary layout transitions almost never happen.
The only situation where unnecessary transitions tend to happen in practice is for swapchain images
that are transitioned from `PresentSrc` to `ColorAttachmentOptimal` before the start of the
render pass, because the initial layout of the render pass attachment is `ColorAttachmentOptimal`
by default for color attachments. Vulkano should make it clear in the documentation of render
passes that the user is encouraged to specify when an attachment is expected to be in the
`PresentSrc` layout.

The only problematic area concerns the first usage of an image, where it must be transitioned from
the `Undefined` or `Preinitialized` layout. This is done by making the user pass a command buffer
builder in the constructor of images, and the constructor adds a transition command to it. The
image implementation is responsible for making sure that the transition command has been submitted
before any further command that uses the image.

# Inter-queue synchronization

When users submit two command buffers to two different queues, they expect the two command buffers
to execute in parallel. However this is forbidden if doing so could result in a data race,
like for example if one command buffer writes to an image and the other one reads from that same
image.
In this situation, the only possible technical solution is to make the execution of the second
command buffer block until the first command buffer has finished executing.
This case is similar to spawning two threads that each access the same resource protected by
a `RwLock` or a `Mutex`. One of the two threads will need to block until the first one is finished.

This raises the question: should vulkano implicitly block command buffers to avoid data races,
or should it force the user to explicitly add wait operations? By comparing a CPU-side
multithreaded program and a GPU-side multithreaded program, then the answer is to make it implicit,
as a CPU will also implicitly block when calling a function that happens to lock a `Mutex` or
a `RwLock`. In CPU code, these locking problems are always "fixed" by properly documenting the
behavior of the functions you call. Similarly, vulkano should precisely document its behavior.

More generally users are encouraged to avoid sharing resources between multiple queues unless these
resources are read-only, and in practice in a video game it is indeed rarely needed to share
resources between multiple queues. Just like for CPU-side multithreading, users are encouraged to
have a graph of the ways queues interact with each other.

However another problem arises. In order to make a command buffer wait for another, you need to
make the queue of the first command buffer submit a semaphore after execution, and the queue of
the second command buffer wait on that same semaphore before execution. Semaphores can only be used
once. This means that when you submit a command buffer to a queue, you must already know if any
other command buffers are going to wait on the one you are submitting, and if so how many. This is not
something that vulkano can automatically determine. The fact that there is therefore no optimal
algorithm for implicit synchronization would be a good point in favor of explicit synchronization.

The decision was taken to encourage users to explicitly handle synchronization between multiple
queues, but if they forget to do so then vulkano will automatically fall back to a dumb
worst-case-scenario but safe behavior. Whenever this dumb behavior is triggered, a debug message
is outputted by vulkano with the `vkDebugReportMessageEXT` function. This message can easily be
caught by the user by registering a callback, or with a debugger.

It is yet to be determined what exactly the user needs to handle. The user will at least need to
specify an optional list of semaphores to signal at each submission, but maybe not the list of
semaphores to wait upon if these can be determined automatically. This has yet to be seen.