We want to get rid of extra command buffers used for staging (and building blases). That would mean tying any command-buffer related things in there to framectl.
However, there are several staging cmdbuf usages which are technically out-of-band wrt framectl:
0. Staging data can get full, which requires sync flush: filling cmdbuf outside of frame (or while still building a frame), submitting it and waiting on it.
1. Texture uploading. There's an explicit usage of staging cmdbuf in vk_texture to do layout transfer. This layout transfer can be moved to staging itself.
2. BLAS building. Creating a ray model uploads its geometry via staging and then immediately builds its BLAS on the same staging cmdbuf. Ideally(?), we'd like to split BLAS building to some later stage to do it in bulk.
1. Fully static (brush model w/o animated textures; studio model w/o animations): singleton, fixed geoms and materials, uploaded only once
2. Semi-static (brush model w/ animated textures): singleton, fixed geoms, may update materials, inplace (e.g. animated textures)
3. Dynamic (beams, triapi, etc): singleton, may update both geoms and materials, inplace
4. Template (sprites): used by multiple instances, fixed geom, multiple materials (colors, textures etc) instances/copies
5. Update-from template (studo models): used by multiple dynamic models, deriving from it wvia BLAS UPDATE, dynamic geom+locations, fixed-ish materials.
API ~
1. RT_ModelCreate(geometries_count dynamic?static?) -> rt_model + preallocated mem
- ~~TODO: updateable: blas[2]? Ping-pong update, cannot do inplace?~~ Nope, can do inplace.
- kusochki
- kusok[]
- geometry -> (vtxidx buffer offsets)
- TODO roughly the same data as VkASBuildRangeInfo, can reuse?
- material (currently embedded in kusok)
- static: tex[], scalar[]
- semi-dynamic:
- (a few) animated tex_base_color
- emissive
- animated with tex_base_color
- individual per-surface patches
- TODO: extract as a different modality not congruent with kusok data
Usage cases for the above:
1. (Fully+semi) static.
- Accept geom[] from above with vtx+idx refernces. Consider them static.
- Allocate static/fixed blas + kusok data once at map load.
- Allocate geom+ranges[] temporarily. Fill them with vtx+idx refs.
- Build BLAS (?: how does this work with lazy/deferred BLAS building wrt geom+ranges allocation)
- Similar to staging: collect everything + temp data, then commit.
- Needs BLAS manager, similar to vk_staging
- Generate Kusok data with current geoms and materials
- Free geom+ranges
- Each frame:
- (semi-static only) Update kusochki materials for animated textures
- Add blas+kusochki_offset (+dynamic color/xform/mmode) to TLAS
2. Preallocated dynamic (triapi)
- Preallocate for fixed N geoms:
- geom+ranges[N].
- BLAS for N geometries
- kusochki[N]
- Each frame:
- Fill geom+ranges with geom data fed from outside
- Fill kusochki --//--
- Fast-Build BLAS as new
- Add to TLAS
3. Dynamic with update (animated studio models, beams)
- When a new studio model entity is encountered:
- Allocate:
- AT FIXED OFFSET: vtx+idx block
- geom+ranges[N], BLAS for N, kusochki[N]
- Each frame:
- Fill geom+ranges with geom data
- Fill kusochki --//--
- First frame: BLAS as new
- Next frames: UPDATE BLAS in-place (depends on fixed offsets for vtx+idx)
- Add to TLAS
4. Instanced (sprites, studio models w/o animations).
- Same as static, BUT potentially dynamic and different materials. I.e. have to have per-instance kusochki copies with slightly different material contents.
- I.e. each frame
- If modifying materials (e.g. different texture for sprites):
- allocate temporary (for this frame only) kusochki block
Go deeply into sequences, animations, etc and figure out whether vertices will actually change.
Might not catch models which are not being animated right now, i.e. current frame is the same as previous one, altough it is not guaranteed to be so.
This potentially conflicts with game logic updating bonetransforms manually even though there are no recorded animations in studio file.
### Detect changes dynamically
Let it process vertices as usual, but then compute hash of vertices values.
Depends on floating point vertices coordinates being bit-perfect same every time, even for moving entities. This is not strictly speaking true because studio model rendering is organized in such a way that bone matrices are pre-multiplied by entity transform matrix. This is done outside of vk_studio.c, and in game dll,which we have no control over. We then undo this multiplication. Given floating point nature of all of this garbage, there will be precision errors and resulting coordinates are not guaranteed to be the same even for completely static models.
### Lazily detect static models, and draw the rest as fully dynamic with fast build
- Detect simple static cases (one sequence, one frame), and pre-build those.
- For everything else, just build it from scratch every frame w/o caching or anything.
If that is not fast enough, then we can proceed with more involved per-entity caching, BLAS updates, cache eviction, etc.
TODO: can we not have a BLAS/model for each submodel? Can it be per-model instead? This would need prior knowledge of submodel count, mesh count, vertices and indices counts. (Potentially conflicts with game dll doing weird things, e.g. skipping certain submodels based on whatever game specific logic)
### Action plan
- [ ] Try to pre-build static studio models. If fails (e.g. still need dynamic knowledge for the first build), then build it lazily, i.e. when the model is rendered for the first time.
- [ ] Needs tracking of model cache entry whenever `m_pStudioHeader` is set.
- [ ] Add a cache for entities, store all prev_* stuff there.
- [ ] Needs tracking of entity cache entry whenever `RI.currententity` is set.
- [ ] Alternative model/entity tracking: just check current ptrs in `R_StudioDrawPoints()` and update them if changed.
- ~~R_DrawStudioModel is the main func for drawing studio model. Called from scene code for each studio entity, with everything current (RI and stuff) set up~~
-`R_StudioDrawModelInternal()` is the main one. It is where it splits into renderer-vs-game rendering functions.
Define new material, independently of any existing textures, etc
This can be .vmat compatible, primext compatbile, etc.
The important parts:
- It has a unique name that we can reference it with
- It has all the fields that we use for our PBR shading model
- (? Material mode can be specified)
```
{
"material" "MAT_NAME"
"map_base_color" "base.png"
"map_normal" "irregular.ktx2"
"base_color" "1 .5 0"
// ...
}
{
"material" "mirror"
"map_base_color" "white"
"base_color" "1 1 1"
"roughness" "0"
"metalness" "1"
// ...
}
```
Then, we can map existing textures to new materials:
```
{
"for_texture" "+EXIT"
"use" "MAT_NAME"
}
```
Or, with more context:
```
{
"for_model_type" "brush"
"for_rendermode" "kRenderTransAlpha"
"for_texture" "wood"
"use" "mat_glass"
"mode" "translucent"
"map_base_color" "glass2.ktx"
}
// ??? meh, see the better _xvk_ example below
{
"for_model_type" "brush"
"for_surface_id" "584"
"use" "mirror"
}
// This example: use previously specified material (e.g. via _xvk stuff below)
// (Depends on applying multiple matching rules, see questions below)
{
"for_model_type" "brush"
"for_rendermode" "kRenderTransAlpha"
"mode" "translucent"
"map_normal" "glass2.ktx"
}
// We also want this (for maps, not globally ofc), see https://github.com/w23/xash3d-fwgs/issues/526
{
"for_entity_id" "39"
"for_texture" "generic028"
"use" "generic_metal1"
}
{
"for_entity_id" "39"
"for_texture" "generic029"
"use" "generic_metal2"
}
```
What it does is:
1. If all `"for_"` fields match, apply values from `"use"` material (in this case `"wood"`)
2. Additionally, override any extra fields/values with ones specified in this block
As we already have surface-patching ability, can just use that for patching materials directly for brush surfaces:
```
// mirror in toilet
{
"_xvk_surface_id" "2057"
"_xvk_material" "mirror"
}
```
Questions:
- Should it apply the first found rule that matches a given geometry and stop?
Or should it apply updates to the material using all the rules that matched in their specified order? Doing the first rule and stopping is more readable and perofrmant, but also might be verbose in some cases.
- Should we do "automatic" materials? I.e. if there's no manually specified material for a texture named `"<TEX>"`, then we try to load `"<TEX>_basecolor.ktx"`, `"<TEX>_normalmap.ktx"`, etc automatically.
Tried refcounts. They only break things, many textures get released prematurely.
Hunch: need to split external/refapi refcount-unaware functionality and hash map into two things:
- old name->gl_texturenum table that is refcount-unaware
- new name->vk.image refcount-aware table and api
# 2023-10-20 E316
## Texture mgmt refcount impedance mismatch: losing textures on changelevel
1. ref_interface_t api is refcount-oblivious: it creates and destroys textures directly. Can't really update refcounts in these calls because they are not balanced at all. Can potentially call Load on the same texture twice, and then delete it only once. (Can it delete the same texture multiple times? Probably not -- need index, which *SHOULD* be inaccessible after the first delte by the API logic -- this is broken now with refcounts)
2. Sequence of events:
1. Changlevel initiated
2. Textures for the old map are deleted using ref_interface api
- mostly us in ref_vk (so we can adjust), but there are a few possible calls from the engine and game.dll
- brings refcount down to "1" (many textures are still referenced from materials)
3. Texture for the new map are created using ref_interface api
- There are common textures that weren't deleted due to being referenced by materials, so they are considered being uploaded already.
- ref_interface_t is refcount-oblivious, so refcounts are not updated, i.e. remaining =1.
4. ref_vk finally notices the changelevel, and starts reloading materials.
- goes through the list of all old materials and releases all the textures
- old textures (but which should've been loaded for the new map too) with refcount=1 are deleted