xash3d-fwgs/ref_vk/TODO.md

14 KiB

Next

  • rtx: configuration that includes texture name -> pbr params mapping, etc. Global, per-map, ...
  • rtx: better light culling: normal, bsp visibility, light volumes and intensity, sort by intensity, etc
  • rtx: simple convolution denoise (bilateral?)
  • rtx: cluster dlights
  • rtx: dynamically sized light clusters Split into 2 buffers: struct LightCluster { uint16 offset, length; } uint8_t data[];

Planned

  • rtx: shrink payload between shaders
  • improve nonuniformEXT usage: https://github.com/KhronosGroup/Vulkan-Samples/pull/243/files#diff-262568ff21d7a618c0069d6a4ddf78e715fe5326c71dd2f5cdf8fc8da929bc4eR31
  • rtx: experiment with refraction index and "refraction roughness"
  • emissive beams
  • emissive particles/sprites
  • issue: transparent brushes are too transparent (train ride)
    • (test_shaders_basic.bsp) shows that for brushes at least there are the following discrepancies with gl renderer:
      • traditional:
        • anything textured transparent is slightly darker in ref_vk
        • "Color" render mode should not sample texture at all and use just color
        • "Texture" looks mostly correct, but ~2x darker than it should be
        • "Glow" looks totally incorrect, it should be the same as "Texture" (as in ref_gl)
        • "Additive" is way too dark in ref_vk
    • rtx: - [ ] "Color" should use solid color instead of texture - [ ] "Color", "Texture", ("Glow"?) should be able to reflect and refract, likely not universally though, as they might be used for different intended effects in game. figure this out on case-by-case basis. maybe we could control it based on texture names and such. - [ ] "Additive" should just be emissive and not reflective/refractive
  • rtx: split ray tracing into modules: pipeline mgmt, buffer mgmt
  • rtx: filter things to render, e.g.: some sprites are there to fake bloom, we don't need to draw them in rtx mode
  • possibly split vk_render into (a) rendering/pipeline, (b) buffer management/allocation, (c) render state
  • rtx: light styles: need static lights data, not clear how and what to do
  • studio models: fix lighting: should have white texture instead of lightmap OR we could write nearest surface lightmap coords to fake light
    • make it look correct lol
  • studio model types:
    • normal
    • float
    • chrome
  • more beams types
  • more particle types
  • rtx: better mip lods: there's a weird math that operates on fov degrees (not radians) that we copypasted from ray tracing gems 2 chapter 7. When the book is available, get through the math and figure this out.
  • sane texture memory management: do not allocate VKDeviceMemory for every texture
  • rtx: transparency layering issue, possible approaches:
    • trace a special transparent-only ray separately from opaque. This can at least be used to remove black texture areas
  • rtx: sky light/emissive skybox:
    • consider baking it into a single (or a few localized) kusok that has one entry in light cluster
    • just ignore sky surfaces and treat not hitting anything as hitting sky. importance-sample by sun direction
    • pre-compute importance sampling direction by searching for ray-miss directions
  • rtx: better memory handling
    • robust tracking of memory hierarchies: global/static, map, frame
    • or just do a generic allocator with compaction?
  • rtx: coalesce all these buffers
  • crash in PM_RecursiveHullCheck
  • rtx: entity lights
  • run under asan
  • rtx: emissive beams
  • rtx: emissive particles
  • rtx: map name to rad files mapping
  • rtx: live rad file reloading (or other solution for tuning lights)
  • rtx: do not rebuild static studio models (most of them). BLAS building takes most of the frame time (~12ms where ray tracing itself is just 3ms)
  • rtx: importance-sample sky light; there are sky surfaces that we can consider light sources
  • cull water surfaces (see c3a2a)
  • create water surfaces once in vk_brush
  • consider doing per-geometry rendermode: brushes can be built only once; late transparency depth sorting for vk render;
  • rtx: too many emissive lights in c3a1b
  • studio models: pre-compute buffer sizes and allocate them at once
  • rtx: denoise
    • non local means ?
    • reprojection
    • SVG+
    • ...
  • rtx: add fps: rasterize into G-buffer, and only then compute lighting with rtx
  • rtx: bake light visibility in compute shader
  • rtx: cull light sources (dlights and light textures) using bsp
  • enable entity-parsed lights by lightstyles
  • dlight for flashlight seems to be broken
  • make 2nd commad buffer for resource upload
  • fix sprite blending; there are commented out functions that we really need (see tunnel before the helicopter in the very beginning)
  • fix projection matrix differences w/ gl render
  • bad condition for temp vs map-permanent buffer error message
  • draw more types of beams
  • fix brush blending
  • sprite depth offset
  • fix incorrect viewport sprite culling
  • improve g_camera handling; trace SetViewPass vs RenderScene ...
  • studio model lighting
  • move all consts to vk_const
  • what is GL_Backend*/GL_RenderFrame ???
  • particles
  • decals
  • render skybox
  • lightmap dynamic styles
  • better flashlight: spotlight instead of dlight point
  • fog
  • studio models survive NewMap; need to compactify buffers after removing all brushes
  • sometimes it gets very slow (1fps) when ran under lldb (only on stream?)
  • optimize perf: cmdbuf managements and semaphores, upload to gpu, ...
  • ? rtx: studio models should not pre-transform vertices with modelView matrix
  • rtx: non-realtime unbiased mode: make "ground truth" screenshots that take 1e5 samples per pixels and seconds to produce. what for: semi-interactive material tuning, comparison w/ denoise, etc.

Someday

  • rtx: dynamic rtx/non-rtx switching breaks dynamic models
  • more than one lightmap texture. E.g. sponza ends up having 3 lightmaps
  • nvnsight into buffer memory and stuff
  • start building command buffers in beginframe
  • multiple frames in flight (#nd cmdbuf, ...)
  • cleanup unused stuff in vk_studio.c
  • embed shaders into binary
  • verify resources lifetime: make sure we don't leak and delete all textures, brushes, models, etc between maps
  • custom allocator for vulkan
  • stats
  • better 2d renderer: fill DRAWQUAD(texture, color, ...) command into storage buffer instead of 4 vertices
  • auto-atlas lots of smol textures: most of model texture are tiny (64x64 or less), can we not rebind them all the time? alt: bindless texture array
  • can we also try to coalesce sprite draw calls?
  • brush geometry is not watertight
  • collect render_draw_t w/o submitting them to cmdbuf, then sort by render_mode, trans depth, and other parameters, trying to batch as much stuff as possible; only then submit

Previously

  • loading to the same map breaks geometry
  • (helps with RTX?) unified rendering (brush/studio models/...), each model is instance, instance data is read from storage buffers, gives info about vertex format, texture bindings, etc; which are read from another set of storage buffers, ..
  • waf shader build step -- get from upstream

2021-02-06

  • alpha test
  • compare w/ gl R_SetRendeMode
    • raster state
    • color constants
  • culling
  • shaders s/map/brush/
  • pipeline cache
  • swapchain getting stale
  • HUD sprites
  • issue: lightmap sometimes gets corrupted on map load

2021-02-08

  • move entity rendering-enumeration into vk_scene

2021-02-10

  • refactor brush into brushes and separate rendering/buffer management
  • animated textures (accept PR)

2021-02-13

  • move pipelines from brush to render
  • render temp buffer api
  • draw studio models somehow
  • studio models vk debug markers
  • studio models white texture as lightmap
  • studio models fixes

2021-02-15

  • weapon models -- viewmodel
  • coalesce studio model draw calls
  • initual sprite support

2021-02-17

  • draw some beams

2021-02-20

  • refactor vk_render interface:
    • move uniform_data_t to global render state ~inside render_draw_t, remove any mentions of uniform/slots from api; alt: global render state?~
    • rename RenderDraw to SubmitDraw
    • ~add debug label to render_draw_t?;~ alt: VK_RenderDebugNameBegin/End
    • perform 3d rendering on corresponding refapi calls, not endframe
  • fix sprite blending

2021-02-22

  • RTX: load extensions with -rtx arg
  • vk_render: buffer-alloc-centric upload and draw api

2021-03-06

  • (RTX; common) Staging vs on-GPU buffers
  • rtx: BLAS construction on buffer unlock
  • rtx: ray trace compute shader
  • dlight test

2021-03-08

  • studio models normals
  • rtx: geometry indexing

2021-03-10

  • rtx: dlights
  • rtx: dlight shadows
  • rtx: dlight soft shadows

2021-03-13

  • rtx: blend normals according to barycentrics
  • rtx: (debug/dev) shader reload
  • rtx: make projection matrix independent render global/current/static state
  • rtx: model matrices
  • rtx: light entities -- still not enough to enlight maps :(
  • rtx: path tracing

2021-03-15

  • rtx: control bounces with cvars
  • rtx: device-local buffers -- doesn't affect perf noticeably :(
  • rtx: emissive materials
    • rtx: emissive textures
    • rtx: emissive beams

2021-03-17..20

  • rtx: lower resolution framebuffer + upscale
  • rtx: importance sample emissive surface
  • rtx: remove entnity-parsed lights
  • rtx: naive temporal denoise: mix with previous frame

2021-03-22

  • rtx: traverse bsp for science!

2021-03-28

  • bake s/d-lights visibility data into bsp leaves

2021-04-06..08

  • persistent models
    • load brushes into render model
    • destroy brushes when time comes (when?)
    • rasterize models in renderer

2021-04-09

  • rtx: build AS for model
  • rtx: include pre-built models in TLAS

2021-04-10

  • rtx: fix tlas rebuild
  • rtx: upload kusochki metadata w/ leaves
  • rtx: add fps
    • rtx: don't group brush draws by texture
    • better AS structure (fewer blases, etc)

2021-04-11

  • vscode build and debug

2021-04-12

  • rtx: fix surface-kusok index mismatch
  • rtx: try to use light visibility data
    • too few slots for light sources
    • some areas have too many naively visible lights
  • rtx: fix light shadow artefacts

2021-04-13

  • rtx: "toilet error": attempting to get AS device address crashes the driver
  • rtx: fix blas destruction on exit
  • rtx: sometimes we get uninitialized models

2021-04-14..16

  • rtx: grid-based light clusters

2021-04-17

  • rtx: read rad file data

2021-04-19

  • rtx: light intensity-based light clusters visibility
  • rtx: check multiple variants of texture name (wad and non-wad)
  • rtx: rad liquids/xeno/... textures

2021-04-22

  • rtx: fix backlight glitch
  • rtx: textures

2021-04-24, E86

  • rtx: restore studio models

2021-05-01, E89

  • make a wrapper for descriptor sets/layouts

2021-05-03, E90

  • make map/frame lifetime aware allocator and use it everywhere: render, rtx buffers, etc

2021-05-08, E92

  • rtx: weird purple bbox-like glitches on dynamic geometry (tlas vs blas memory corruption/aliasing)
  • rtx: some studio models have glitchy geometry

2021-05-10, E93

  • rtx: don't recreate tlas each frame
  • rtx: dynamic models AS caching

2021-05-..-17, E93, E94

  • rtx: improve AS lifetime/management; i.e. pre-cache them, etc
  • add debug names to all of the buffers

2021-05-22, E97

  • add nvidia aftermath sdk

2021-05-24, E98

  • rtx: simplify AS tracking

2021-05-26, E99

  • rtx: fix device lost after map load

2021-05-28, E100

  • rtx: build acceleration structures in a single queue/cmdbuf

2021-06-05, E103

  • rtx: dynamic surface lights / dynamic light clusters
  • rtx: animated textures
  • rtx: attenuate surface lights by normal

2021-06-07, E104..

  • fix CI for vulkan branch

2021-06-09..12, E105..106

  • c3a2a: no water surfaces in vk (transparent in gl: *45,*24,*19-21)
  • water surfaces

2021-06-14, E107

  • rtx: optimize water normals. now they're very slow because we R/W gpu mem? yes
  • cull bottom water surfaces (they're PLANE_Z looking down)
  • fix water normals

2021-06-23, E109

  • rtx: ray tracing shaders specialization, e.g. for light clusters constants
  • rtx: restore dynamic stuff like particles, beams, etc
  • rtx: c3a1b: assert model->size >= build_size.accelerationStructureSize failed at vk_rtx.c:347

2021-07-17, E110..120

  • rtx: ray tracing pipeline
  • rtx: fix rendering on AMD
  • rtx: split models into a separate module
  • rtx: alpha test

2021-07-31, E121

  • rtx: alpha blending -- did a PoC

2021-08-02..04, E122-123

  • mipmaps
  • rtx: better random

2021-08-07, E124

  • anisotropic texture sampling
  • studio model lighting prep
    • copy over R_LightVec from GL renderer
    • add per-vertex color attribute
    • support per-vertex colors
    • disable lightmaps, or use white texture for it instead

2021-08-11, E125

  • simplify buffer api: do alloc+lock as a single op

2021-08-15, E126

  • restore render debug labels
  • restore draw call concatenation; brush geoms are generated in a way that makes concatenating them impossible

2021-08-16, E127

  • better device enumeration

2021-08-18, E128

  • rtx: fix maxVertex for brushes

2021-08-22, E129

  • fix depth test for glow render mode
  • screenshots

2021-08-26, E131

  • rtx: material flags for kusochki

2021-09-01, E132

  • rtx: ingest brdfs from ray tracing gems 2
  • rtx: directly select a triangle for light sampling

2021-09-04, E133

  • rtx: different sbts for opaque and alpha mask
  • include common headers with struct definitions from both shaders and c code

2021-09-06, E134

  • rtx: pass alpha for transparency
  • rtx: remove additive/refractive flags in favor or probability of ray continuing further instead of bouncing off
  • make a list of all possible materials, categorize them and figure out what to do