Too many places in the engine and the renderer expect texture names to
be insensitive: RAD files, material references, probably engine and game
calls too.
Make it universally case insensitive. Also, add rudimentary tests for
it.
Previous hash table was case insensitive, while the new one is
sensitive.
For now the workaround is to tolower texture names in rad files.
Proper way would be to address case-sensitivity globally. Currently it's
not very consistent.
Known issues:
- pbr materials are completely broken. They end up not being able to
find textures, seemingly due to memory corruption on materials side,
not textures.
- There are still places that try to get texture=1
Adds unordered_roadmap simple hash map:
- open addressing with linear probing
- size is fixed at init/compile time
- operates on an pre-allocated array of items with hashmap headers
Also adds basic tests for it.
And properly enables tests for ref_vk (i.e. alolcator)
THIS COMMIT IS BROKEN AND LEADS TO MISSING TEXTURES
There's a "race" between texture release/acquire on changelevel.
Textures end up being deleted when they shouldn't.
Addressing this is tedious, will be done in the following commits.
* engine: common: imagelib: add KTX2 support
Adds basic KTX2 support for a few compressed formats. KTX2 essentially
is a Vulkan-centric texture format that supports literally hundreds of
pixel formats.
For now only support for these is added:
- `VK_FORMAT_BC4_UNORM_BLOCK`
- `VK_FORMAT_BC4_SNORM_BLOCK`
- `VK_FORMAT_BC5_UNORM_BLOCK`
- `VK_FORMAT_BC5_SNORM_BLOCK`
- `VK_FORMAT_BC6H_UFLOAT_BLOCK`
- `VK_FORMAT_BC6H_SFLOAT_BLOCK`
- `VK_FORMAT_BC7_UNORM_BLOCK`
- `VK_FORMAT_BC7_SRGB_BLOCK`
Adding more formats is relatively straightforward:
- Copy format definition from `VkFormat` enum in `vulkan_core.h`
- Add a new definition into `pixformat_t` enum.
- Add format size calculation into `Image_ComputeSize()`
While we're at it, also adds a few new formats to DDS:
- BC4_UNORM -- PF_BC4_UNSIGNED
- BC4_SNORM -- PF_BC4_SIGNED
- BC5_UNORM -- PF_BC5_UNSIGNED
- BC5_SNORM -- PF_BC5_SIGNED
- BC7 is expanded into BC7_UNORM and BC7_SRGB
ref_gl and ref_soft code is updated where it made sense. But not tested
really. Support for these formats has been tested with ref_vk.
* address spaces-vs-parentheses formatting where noticed
* parenthesize sizeofs
* move ktx2.h to imagelib as img_ktx2.h; massage it a bit
* use SetBits() instead of |=
* remove stale TODO comments
We were running out of descriptor sets, as they weren't being freed at all.
Now there's a 1-1 table of texture-descset, and no way to run out of them.
Better solution would be to move this descset management to vk_render, and allocate them dynamically from a smaller pool.
Support DDS and KTX2
- [x] Pass KTX2 as raw data
- [x] Simplify image uploading (there are 2 copies of the same uploading code for regular textures and for KTX2)
- [x] Extract long format lists from `vk_image.c` to e.g. `vk_image_extra.h`
- [x] Support DDS formats
- [x] BC7
- [x] PoC works
- [x] read supplied mips
- [x] Pass most common KTX2 subformats as correct PF_ format
- [x] BC5
- [x] BC6H
- [x] BC7 -- need UNORM/SRGB expansion
- [x] BC4 -- need to add it
- [x] Alpha flag for KTX2 formats
- [x] swizzle r->rgba
- [x] check il_dds_hardware
- [x] add `IL_KTX2_RAW_SUPPORTED`
- [x] format imagelib things
- [x] extract image size function to img_utils
Fixes#154
It only does a very basic header validation, and passes the entire file
as PF_KTX2_RAW format. This is to simplify KTX2 reading in ref_vk and
trying out different formats. KTX2 has support for >200 format, and
passing all of them through PF_ types is a non-starter.
The plan is to figure out which formats we want to support, and add
their support to imagelib/ktx2 incrementally, leaving the rest as
PF_KTX2_RAW, so ref_vk still can use them.
`"inherit" "debug_test"` will replace all fields for the current
material with ones from material named "debug_test".
`"inherit"` should be specified before any material fields, except for
selectors like "new", "for" and "for_rendermode".
Storing it linearly was a mistake: it is 8-bit only, and lacks enough
precision for dark values. It also doesn't really need any more
precision, and should be limited to 0..1 range.
Therefore, it makes sense to treat it as sRGB explicitly.
Traditional renderer operates in sRGB-γ space (as everyone was in 199x).
Making vulkan textures explicitly SRGB for RT renderer breaks color for
trad rendere. Make sure trad renderer still has raw SRGB-unaware texture
sampling.
Adds an UNORM image view for all relevant SRGB textures, and uses them
exclusively for trad renderer. RT still gets proper SRGB views.
Also rename XVK_ to R_Vk for images
Make sure that all SRGB-gamma textures are marked with SRGB format (this
includes all the original textues, 8bit PNG textues). All other
textures (roughness, metallic, normal maps) are linear (UNORM).
Trust KTX2 texture that they store in the correct colorspace.
Then we can just sample textures w/o SRGBtoLINEAR'izing them. Supposedly
this is better because of hw interpolation in the correct colorspace.
This is a bit experimental because Compressonator can't really into
BC7_SRGB, which is sad. And also we need to re-gamma all the textures
for the gamma/srgb-unaware traditional renderer.
They have slightly different incompatible encodings. Switch between them
using #define at build time.
Old png code can be removed when we fully switch to KTX2.
Loads something. Not everything. Not everything correctly.
Doesn't do validity checks.
Makes colorspace (srgb gamma vs linear) inconsistent between textures
(ktx has correct srgb-vs-unorm, all is unorm (but srgb gamma
semantically) in png)
Makes normals inconsistent too (.xy in ktx, .xyz in png)
related #154
Was triggered by renormalizing normal z when applying normal scale.
Not sure why exactly, though, the math checks out.
Doing it a bit differently solves the issue.
Fixes#595
kRenderTransColor mode strips geometry of its texture and makes it solid
color. Previously we were noticing this rendermode late, and failed to
update the textures.
Now we depend on reading this rendermode from map entities, and
pre-assigning correct (white) textures on load.
Assumes that rendermode doesn't change at runtime for brush models.
Fixes#528
- [x] Pass material reference via geom/render API
- [x] Show original textures for traditional renderer, #571
- [x] Remove old material type flags
- [x] ~~Move sky surfaces to a separate BLAS~~ Remove `SURF_DRAWSKY` from geometries completely #474
- [x] **REVERT** `SURF_DRAWSKY` changes. Such surfaces still need to be "drawn" to hide geometry behind it (see comments).
- [x] Chrome material: have an explicit material for it
- [x] This needs vk_texture vs vk_material index decoupling
- [x] Assess #577
- [x] Improve paths #578
- [x] `pbr/maps/c0a0a.bsp/c0a0a.bsp.mat` -> `pbr/maps/c0a0a.bsp/c0a0a.mat`
- [x] `pbr/halflife.wad/halflife.wad.mat` -> `pbr/halflife.wad/halflife.mat`
- [x] Rename files in PBR repo for the above
- [x] `luchiki/maps/c0a0.bsp.patch` -> `luchiki/maps/c0a0.patch`
- [x] Rename patch files too
- [x] Same for spr,mdl
- [x] Rename
- [x] Pass material mode directly, instead of render type
- [x] Advanced material patching/selection
- [x] Need a few advanced selection examples
- [x] #317
- [x] стёкла у костюма (??)
- [x] #526
- [x] #577
- [x] #213
Different render sources (model types, render types, etc) might require
different material modes. E.g. brush should be translucent (refractive+mirror)
for blend mix modes. However, smoke particles should not be
mirror/refractive for the same blend mix render type.
Make struct `vk_devmem_allocation_stats_t` which will be able to store
overall stats and stats generic for each usage type.
Move Vulkan related devmem allocation arguments into their own type -
`vk_devmem_allocate_args_t`.
Store `vk_devmem_usage_type_t` inside `vk_devmem_t`, so we can handle
deallocations (frees) for each corresponding type aside from overall
stats.
Make `VK_DevMemAllocateBuffer` and `VK_DevMemAllocateImage` macros which
call generic function `VK_DevMemAllocate` with correspoding usage type
set.
Make function `VK_DevMemUsageTypeString` to be able to get short string
of long enum name.
Needed to hide invisible geometry on e.g. c5a1. Breaks de_dust2 roofs
(#474). That will need to be addressed some other way.
Still uses special value for base_color texture to signal SKY SURFACE.
A supposedly better way to do this would be to have them have a special
material.
Use built-in `Q_memprint` to output memory sizes.
Use built-in `COM_FileWithoutPath` instead of newly added
`get_filepath_from_filepath` to truncate full filepath from filename.
Clarify type name `vk_device_memory_t` as `vk_device_memory_slot_t`
to not confuse it with `vk_devmem_t` as their names are pretty much
the same.
Add additional helper functions to standardize VK flags output format.
Also maybe it is a good idea to have it inside `vk_flags.{h,c}`,
for example.
Improve debug printing: bring colors and improve readability.
Add internal (private) field `_block_size` to `vk_devmem_t` so we can
keep track of how much memory we freed.
1. Completely remove them from geometries
2. Draw skybox where the ray hasn't hit anything
3. In the same fashion sky is not shadowed when there's nothing hit by
the shadow ray.
This allows completely removing the sky material flag.
Also supposedly fixes#474
This is needed to enable advanced material patching, where the material
can be picked up based on source type, render mode, surface number, etc
etc and all of the above combined if needed.
Previous scheme was picking up materials very late when all of this info
has been already lost.
Remove unnecessary `crtlib.h` import in `profiler.h`.
Simplify `get_filename_from_filepath` function.
Add extra guards in metrics line printing to make sure we would not
leak/blow up anything.
Previously, source filepath was truncated right at metric registration,
so only the filename was stored. Now, full source filepath is stored.
The truncation to its filename happens only in metrics print. This way
we preserve full information, but also throw away redundancy in print.
Remove command `r_speeds_list_metrics` and cvar
`r_speeds_metrics_as_table`. Now, there are 2 new commands:
`r_speeds_mlist` - to print metrics as a list, and
`r_speeds_mtable` - to print metrics as a table.
Both of them can handle optional filter argument.
Reduces enormous amount of space used by absolute filepaths in
metrics print. This does not mean we cannot locate the files now.
Pretty much all of the vk files have such prefix in their name.
Slightly improve metrics list print formatting and
add new option to print it as a table. Table alignment relies on
monospace font.
This can be turned on/off with cvar `r_speeds_metrics_as_table`.
Fixing pool allocator to properly signal allocation failure uncovered an
existing issue where we were lacking enough memory for dynamic model
BLASes on Linux/amdgpu. Erroneously the same memory region was used for
>1 BLAS. Surprisingly this hasn't led to any noticeable issues so far.
Increasing accels buffer size fixes the issue.
Entity data gets reallocate on every NewMap regardless of whether it was a load from a save. Thus all entity pointers and data are invalidated. EntityData keeps things using pointers as keys.
Make sure it is properly cleared on `R_NewMap()`
Fixes#562
Returning `0` instead of `ALO_ALLOC_FAILED` let to API consumers believe that allocation was successful. This lead to asserts failing when trying to free such allocations.
Makes #562 not crash (but miss studio models anyway)
python/357 has bullets submodels which are animated using parent bones.
Their direct bones are static, and thus the entire submodel was picked
up as static.
Now when computing bone transform for particular sequence/anim frame
also merge it with parent transform. Bones are laid out sequentially in
their "dependency order" so using a direct parent is fine, as it also
contains its parent transforms.
Fixes: #554
On Windows we're seeing a max size = 0x0 swapchains. Those cannot be created or used. Make sure that we're not, and we're not trying to draw anything when there's no swapchain available.
Unfortunately we still have to call some rendering functions (without actually rendering anything) to make sure that various invariants hold.
fixes#463
They are not used by anything yet.
Fallback to generating regular noise textures if bluenoise ones weren't
found.
Real blue noise textures require Half-Life-PBR repo textures.
Do not aggravate the validator by uploading to the same memory region twice. Make sure that there's synchronization between updating kusochki.
Fixes#553
Sometimes the initial list of models doesn't contain everything that is
used later. I don't know why. So detect this and allow loading model
infos later.
Fixes#546
This doesn't work as expected in some cases, as surfaces might still get
linked transitively by neighbours. Solving this seems non-trivial for
now, but maybe we can just live with it
Automatically smooth normals between surfaces with normals less than 45
degrees off.
Can be adjusted from map.bsp.patch file like this (e.g. to 50 degrees):
```
{
"_xvk_smoothing_threshold" "50"
}
```
Add render refcount to submodels to diagnose when it can't properly clear things due to them being used somewhere still.
Also add `speeds.submodels_cache_{dynamic,static}` counters to show how many submodels render models have been allocated.
Gives out individual render submodels for dynamic ones, receives and remembers them when they are not needed anymore.
Stores only one render submodel for static ones. Reuses/instantiates it for everyone.
Detect whether studio model has changed, and replace it with a new one.
It does (as predicted) run into a gpu sync issue if reuse happened while previous frame with the old user is still being drawn.
Renames previous METRICS to COUNTERS. These are still reset to zero
every frame.
Adds new METRICS which are preserved, maintained externally to speeds,
and only sampled by speeds code once per frame.
Also adds new metrics:
- `studio.cached_submodels` -- number of submodels in cache
- `geom.used` -- memory used by long allocations
- `geom.{vertices,indices}` -- counts of vertices/indices for long
allocations
- `geom.dyn_{vertices,indices}` -- counts of vertices/indices for
single-frame dynamic allocations
When `r_speeds_graphs` cvar is read for the first time, not all metrics
might be registered yet. It leads to some graphs missing.
Retry searching for these metrics on further frames, thus preserving the
graph list from previous session fully.
- Add variable name and registration src:line to the
`r_speeds_list_metrics` output. Makes it easier to reason about where
does this metric come from.
- Group metrics by their modules, makes it easier to discover.
- Do not print the list immediately on command, do it later in the
frame. Makes it print correct latest frame values.
Still slightly KORYAVY as:
- there's no cache eviction, it might get full pretty quick
- static-vs-dynamic animation decision is pretty rudimentary, might consider non-animated things as dynamic
- args passing vs global state is meh
This makes water static for now, and also makes it look weird, e.g.:
- no culling of water sides (it is "dynamic" in a sense that it's an
entity property, not model property)
Fixes computing total vertices/indices count. Were referencing the same `pmesh` for all meshes.
Floating and missing heads issue seems to be due to incorrect fixed animation frames. I.e. animation frames contain offsets to the correct positions. Should be fixed when animations are done.
Known issues:
- no animations, stuck at first rendered frame of a given submodel.
This is as intended for now, needs BLAS update functionality not yet implemented.
- wrong positions/transforms. Reasons unknown. May be "as intended" too.
- missing heads
- Barney model corruption. Consistent between maps/instances/animation
frames.
Somehow makes sprites correctly oriented (and winding-culled) for ray
tracing, but makes them disappear (winding-culled) for traditional
renderer. This makes zero sense.
This is a workaround for inconsistent memory management: some subsystems completely clear their memories and start allocating from zero. This leads to overwriting long lived sprite model with garbage.
TODO FIXME
Make sure that things get created and destroyed at the right times.
Allow longer-than-map block allocations.
Fix brush model leaks -- previously they weren't destroyed on map
change/game exit properly. Also free geometry ranges accordingly.
Add a note about map loading process, and various models lifetimes.
The intent is to manage long-vs-single-frame allocations better.
Previously long allocations were map-long bump allocations, and couldn't be freed
mid-map, as there was neither a reference to the allocated range, nor a
way to actully free it.
Add a two-mode block allocator (similar to previous debuffer alloc) that
allows making long and once allocations. But now long allocations are
backed by "pool" allocator and return references to the range.
This commit doesn't do the deallocation yet, so map chaning doesn't yet
work.
- explicitly group cache-related fields
- move kusochki allocation to where it's actually used
this makes a step towards better blas management from bottom up
Draft the new accel/blas apis. Consolidate everything accel-related into
vk_ray_accel.c. Start splitting into more atomic functions. Prepare for
blas-model+kusochki split. etc etc.
The new code isn't really used yet.
1. Rename models passed to TLAS to instances.
2. Remove BLAS validation: old, doesn't make sense anymore.
3. Draft general blas mgmt approach in NOTES.md
Still has some artifacts, but is generally rather close. Will look at
this again when caching studio model BLASes, as we'd be able to look at
them without extra animations.
- We're not going to implement R_ShowTextures for ref_vk, at least in
its gl form.
- It seems that Mod_GetCurrentVis() can just return NULL, at least
ref_soft does it.
Fixes#269, fixes#254
Ivan Avdeev
Alibek Omarov
mittorn
Ivan Avdeev
mittorn
Valery Klachkov
NightFox
nilsoncore
Velaron
fgsfds
Ivan Avdeev
