2022-09-30 10:05:07 +02:00
# Programmable render
- [ ] parse spirv -> get bindings with names
- [ ] spirv needs to be compiled with -g, otherwise there are no OpName entries. Need a custom strip util that strips the rest?
- [ ] unnamed uniform blocks are uncomfortable to parse.
- [ ] passes "export" their bindings as detailed resource descriptions:
- [ ] images: name, r/w, format, resolution (? not found in spv, needs to be externally supplied)
- [ ] buffers: name, r/w, size, type name (?)
- [ ] name -> index resolver (hashmap kekw)
- [ ] automatic creation of resources
- [ ] images
- [ ] buffers
- [ ] implicit dependency tracking. pass defines:
- [ ] imports: list of things it needs
- [ ] exports: list of things it produces. those get created and registered with this pass as a producer
# Parallel frames sync
- [ ] light_grid_buffer (+ small lights_buffer):
2022-05-29 09:01:43 +02:00
- lifetime: single frame
- BUT: populated by CPU, needs sync; can't just ring-buffer it
- fixes: double-buffering?
- staging + sync upload? staging needs to be huge or done in chunks. also, cpu needs to wait on staging upload
- 2x size + wait: won't fit into device-local-host-visible mem
- decrease size first?
2022-09-30 10:05:07 +02:00
- [ ] models_cache
2022-05-29 09:01:43 +02:00
- lifetimes:
- static; entire map
- static; single to multiple frames
- dynamic; multiple frames
- : intra-cmdbuf
2022-09-30 10:05:07 +02:00
# Multipass + Sampling
2022-02-05 03:38:36 +01:00
- [ ] better simple sampling
2022-02-10 06:57:55 +01:00
- [x] all triangles
- [x] area based on triangles
2022-02-05 03:38:36 +01:00
- [ ] clipping?
- [ ] can we pack polygon lights better? e.g.:
- each light is strictly a triangle
- index is offset into triangles
- layout:
- vec4(plane) // is it really needed? is early culling important? can we shove area into there too? e.g plane_n.xy,plane_d, area
- vec4(v0xyz, e_r)
- vec4(v1xyz, e_g)
- vec4(v2xyz, e_b)
- [ ] additive transparency
- [ ] bounces
- [ ] skybox shadows
2021-10-26 19:40:02 +02:00
# Next
- [ ] remove surface visibility cache
2021-10-24 18:58:41 +02:00
- [ ] rtx: rename point lights to lampochki
2021-10-24 20:21:22 +02:00
- [ ] rtx: rename emissive surface to surface lights
2021-04-22 19:42:19 +02:00
- [ ] rtx: dynamically sized light clusters
Split into 2 buffers:
struct LightCluster { uint16 offset, length; }
uint8_t data[];
2021-02-06 20:07:00 +01:00
# Planned
2021-09-01 19:37:30 +02:00
- [ ] rtx: shrink payload between shaders
2021-08-26 18:18:20 +02:00
- [ ] improve nonuniformEXT usage: https://github.com/KhronosGroup/Vulkan-Samples/pull/243/files#diff-262568ff21d7a618c0069d6a4ddf78e715fe5326c71dd2f5cdf8fc8da929bc4eR31
2021-09-06 23:14:59 +02:00
- [ ] rtx: experiment with refraction index and "refraction roughness"
2021-08-26 18:18:20 +02:00
- [ ] emissive beams
- [ ] emissive particles/sprites
2021-08-22 19:44:47 +02:00
- [ ] issue: transparent brushes are too transparent (train ride)
- [ ] (test_shaders_basic.bsp) shows that for brushes at least there are the following discrepancies with gl renderer:
- [ ] traditional:
- [ ] anything textured transparent is slightly darker in ref_vk
- [ ] "Color" render mode should not sample texture at all and use just color
- [ ] "Texture" looks mostly correct, but ~2x darker than it should be
- [ ] "Glow" looks totally incorrect, it should be the same as "Texture" (as in ref_gl)
- [ ] "Additive" is way too dark in ref_vk
- [ ] rtx:
- [ ] "Color" should use solid color instead of texture
- [ ] "Color", "Texture", ("Glow"?) should be able to reflect and refract, likely not universally though, as they might be used for different intended effects in game. figure this out on case-by-case basis. maybe we could control it based on texture names and such.
- [ ] "Additive" should just be emissive and not reflective/refractive
- [ ] rtx: split ray tracing into modules: pipeline mgmt, buffer mgmt
- [ ] rtx: filter things to render, e.g.: some sprites are there to fake bloom, we don't need to draw them in rtx mode
2021-08-11 20:43:33 +02:00
- [ ] possibly split vk_render into (a) rendering/pipeline, (b) buffer management/allocation, (c) render state
- [ ] rtx: light styles: need static lights data, not clear how and what to do
- [ ] studio models: fix lighting: should have white texture instead of lightmap OR we could write nearest surface lightmap coords to fake light
- [ ] make it look correct lol
2021-08-08 04:56:38 +02:00
- [ ] studio model types:
- [x] normal
- [ ] float
2021-12-22 06:37:36 +01:00
- [x] chrome
2021-08-08 04:56:38 +02:00
- [ ] more beams types
- [ ] more particle types
2021-08-05 03:36:53 +02:00
- [ ] rtx: better mip lods: there's a weird math that operates on fov degrees (not radians) that we copypasted from ray tracing gems 2 chapter 7. When the book is available, get through the math and figure this out.
- [ ] sane texture memory management: do not allocate VKDeviceMemory for every texture
2021-07-31 21:43:35 +02:00
- [ ] rtx: transparency layering issue, possible approaches:
- [ ] trace a special transparent-only ray separately from opaque. This can at least be used to remove black texture areas
2021-07-17 21:37:12 +02:00
- [ ] rtx: sky light/emissive skybox:
- [ ] consider baking it into a single (or a few localized) kusok that has one entry in light cluster
- [ ] just ignore sky surfaces and treat not hitting anything as hitting sky. importance-sample by sun direction
- [ ] pre-compute importance sampling direction by searching for ray-miss directions
2021-06-23 20:37:54 +02:00
- [ ] rtx: better memory handling
- [ ] robust tracking of memory hierarchies: global/static, map, frame
- or just do a generic allocator with compaction?
- [ ] rtx: coalesce all these buffers
- [ ] rtx: entity lights
2021-06-19 19:51:21 +02:00
- [ ] rtx: do not rebuild static studio models (most of them). BLAS building takes most of the frame time (~12ms where ray tracing itself is just 3ms)
2021-06-12 21:54:53 +02:00
- [ ] rtx: importance-sample sky light; there are sky surfaces that we can consider light sources
- [ ] cull water surfaces (see c3a2a)
- [ ] create water surfaces once in vk_brush
- [ ] consider doing per-geometry rendermode: brushes can be built only once; late transparency depth sorting for vk render;
2021-06-05 22:06:03 +02:00
- [ ] rtx: too many emissive lights in c3a1b
2021-05-17 19:57:16 +02:00
- [ ] studio models: pre-compute buffer sizes and allocate them at once
2021-04-12 20:07:41 +02:00
- [ ] rtx: denoise
- [ ] non local means ?
- [ ] reprojection
- [ ] SVG+
- [ ] ...
2021-04-10 21:46:37 +02:00
- [ ] rtx: add fps: rasterize into G-buffer, and only then compute lighting with rtx
2021-04-24 21:53:42 +02:00
- [ ] rtx: bake light visibility in compute shader
2021-03-14 01:33:17 +01:00
- [ ] dlight for flashlight seems to be broken
2021-02-20 23:54:57 +01:00
- [ ] make 2nd commad buffer for resource upload
- [ ] fix sprite blending; there are commented out functions that we really need (see tunnel before the helicopter in the very beginning)
2021-02-20 21:00:31 +01:00
- [ ] fix projection matrix differences w/ gl render
- [ ] bad condition for temp vs map-permanent buffer error message
2021-02-17 22:26:09 +01:00
- [ ] fix brush blending
- [ ] sprite depth offset
- [ ] fix incorrect viewport sprite culling
2021-02-15 21:20:25 +01:00
- [ ] improve g_camera handling; trace SetViewPass vs RenderScene ...
2021-02-15 18:55:42 +01:00
- [ ] studio model lighting
2021-02-08 19:57:27 +01:00
- [ ] move all consts to vk_const
2021-02-06 22:13:08 +01:00
- [ ] what is GL_Backend*/GL_RenderFrame ???
- [ ] particles
- [ ] decals
2021-02-06 20:07:00 +01:00
- [ ] render skybox
2021-02-06 20:38:08 +01:00
- [ ] lightmap dynamic styles
2021-07-17 21:37:12 +02:00
- [ ] better flashlight: spotlight instead of dlight point
2021-02-10 19:33:44 +01:00
- [ ] fog
- [ ] studio models survive NewMap; need to compactify buffers after removing all brushes
2021-02-15 18:55:42 +01:00
- [ ] sometimes it gets very slow (1fps) when ran under lldb (only on stream?)
2021-02-17 22:26:09 +01:00
- [ ] optimize perf: cmdbuf managements and semaphores, upload to gpu, ...
2021-09-04 21:36:30 +02:00
- [ ] ? rtx: studio models should not pre-transform vertices with modelView matrix
- [ ] rtx: non-realtime unbiased mode: make "ground truth" screenshots that take 1e5 samples per pixels and seconds to produce. what for: semi-interactive material tuning, comparison w/ denoise, etc.
2021-02-06 20:07:00 +01:00
# Someday
2021-06-23 19:40:37 +02:00
- [ ] more than one lightmap texture. E.g. sponza ends up having 3 lightmaps
2021-05-17 19:57:16 +02:00
- [ ] nvnsight into buffer memory and stuff
2021-02-20 23:54:57 +01:00
- [ ] start building command buffers in beginframe
- [ ] multiple frames in flight (#nd cmdbuf, ...)
2021-02-15 21:20:25 +01:00
- [ ] cleanup unused stuff in vk_studio.c
2021-02-06 20:38:08 +01:00
- [ ] embed shaders into binary
2021-02-06 22:13:08 +01:00
- [ ] verify resources lifetime: make sure we don't leak and delete all textures, brushes, models, etc between maps
- [ ] custom allocator for vulkan
- [ ] stats
- [ ] better 2d renderer: fill DRAWQUAD(texture, color, ...) command into storage buffer instead of 4 vertices
2021-02-15 18:55:42 +01:00
- [ ] auto-atlas lots of smol textures: most of model texture are tiny (64x64 or less), can we not rebind them all the time? alt: bindless texture array
2021-02-15 21:20:25 +01:00
- [ ] can we also try to coalesce sprite draw calls?
2021-08-15 20:02:57 +02:00
- [ ] brush geometry is not watertight
2021-02-20 08:07:45 +01:00
- [ ] collect render_draw_t w/o submitting them to cmdbuf, then sort by render_mode, trans depth, and other parameters, trying to batch as much stuff as possible; only then submit
2021-02-17 22:26:09 +01:00
2021-02-23 03:54:13 +01:00
# Previously
2021-07-17 21:37:12 +02:00
- [x] loading to the same map breaks geometry
- [x] (helps with RTX?) unified rendering (brush/studio models/...), each model is instance, instance data is read from storage buffers, gives info about vertex format, texture bindings, etc; which are read from another set of storage buffers, ..
- [x] waf shader build step -- get from upstream
2021-02-08 19:57:27 +01:00
## 2021-02-06
- [x] alpha test
- [x] compare w/ gl R_SetRendeMode
- [x] raster state
- [x] color constants
- [x] culling
- [x] shaders s/map/brush/
- [x] pipeline cache
- [x] swapchain getting stale
- [x] HUD sprites
- [x] issue: lightmap sometimes gets corrupted on map load
2021-02-10 19:33:44 +01:00
## 2021-02-08
- [x] move entity rendering-enumeration into vk_scene
2021-02-14 02:19:59 +01:00
## 2021-02-10
- [x] refactor brush into brushes and separate rendering/buffer management
- [x] animated textures (accept PR)
## 2021-02-13
- [x] move pipelines from brush to render
- [x] render temp buffer api
- [x] draw studio models somehow
- [x] studio models vk debug markers
- [x] studio models white texture as lightmap
2021-02-15 18:55:42 +01:00
- [x] studio models fixes
2021-02-17 22:26:09 +01:00
## 2021-02-15
- [x] weapon models -- viewmodel
- [x] coalesce studio model draw calls
- [x] initual sprite support
## 2021-02-17
- [x] draw some beams
2021-02-23 03:54:13 +01:00
## 2021-02-20
- [x] refactor vk_render interface:
- [x] move uniform_data_t to global render state ~inside render_draw_t, remove any mentions of uniform/slots from api; alt: global render state?~
- [x] rename RenderDraw to SubmitDraw
- [x] ~add debug label to render_draw_t?;~ alt: VK_RenderDebugNameBegin/End
- [x] perform 3d rendering on corresponding refapi calls, not endframe
- [x] fix sprite blending
## 2021-02-22
- [x] RTX: load extensions with -rtx arg
- [x] vk_render: buffer-alloc-centric upload and draw api
2021-03-08 18:54:26 +01:00
## 2021-03-06
- [x] (RTX; common) Staging vs on-GPU buffers
2021-03-10 19:13:59 +01:00
- [x] rtx: BLAS construction on buffer unlock
- [x] rtx: ray trace compute shader
2021-03-08 18:54:26 +01:00
- [x] dlight test
2021-03-10 19:13:59 +01:00
## 2021-03-08
- [x] studio models normals
- [x] rtx: geometry indexing
2021-03-13 19:46:25 +01:00
## 2021-03-10
- [x] rtx: dlights
- [x] rtx: dlight shadows
- [x] rtx: dlight soft shadows
2021-03-15 17:44:15 +01:00
## 2021-03-13
- [x] rtx: blend normals according to barycentrics
- [x] rtx: (debug/dev) shader reload
- [x] rtx: make projection matrix independent render global/current/static state
- [x] rtx: model matrices
- [x] rtx: light entities -- still not enough to enlight maps :(
- [x] rtx: path tracing
2021-03-17 17:15:33 +01:00
## 2021-03-15
- [x] rtx: control bounces with cvars
- [x] rtx: device-local buffers -- doesn't affect perf noticeably :(
- [x] rtx: emissive materials
- [x] rtx: emissive textures
- [x] rtx: emissive beams
2021-03-28 22:52:25 +02:00
## 2021-03-17..20
- [x] rtx: lower resolution framebuffer + upscale
- [x] rtx: importance sample emissive surface
- [x] rtx: remove entnity-parsed lights
- [x] rtx: naive temporal denoise: mix with previous frame
## 2021-03-22
- [x] rtx: traverse bsp for science!
2021-04-07 21:11:20 +02:00
## 2021-03-28
- [x] bake s/d-lights visibility data into bsp leaves
2021-04-10 19:58:46 +02:00
## 2021-04-06..08
- [x] persistent models
- [x] load brushes into render model
- [x] destroy brushes when time comes (when?)
- [x] rasterize models in renderer
## 2021-04-09
- [x] rtx: build AS for model
- [x] rtx: include pre-built models in TLAS
2021-04-10 21:46:37 +02:00
## 2021-04-10
- [x] rtx: fix tlas rebuild
- [x] rtx: upload kusochki metadata ~~w/ leaves~~
- [x] rtx: add fps
- [x] rtx: don't group brush draws by texture
- [x] better AS structure (fewer blases, etc)
2021-04-12 20:07:41 +02:00
## 2021-04-11
- [x] vscode build and debug
2021-04-13 19:13:17 +02:00
## 2021-04-12
- [x] rtx: fix surface-kusok index mismatch
- [x] rtx: try to use light visibility data
2021-09-06 23:14:59 +02:00
- too few slots for light sources
- some areas have too many naively visible lights
2021-04-13 19:13:17 +02:00
- [x] rtx: fix light shadow artefacts
2021-04-16 20:43:37 +02:00
## 2021-04-13
- [x] rtx: "toilet error": attempting to get AS device address crashes the driver
- [x] rtx: fix blas destruction on exit
- [x] rtx: sometimes we get uninitialized models
2021-04-17 21:33:46 +02:00
## 2021-04-14..16
- [x] rtx: grid-based light clusters
## 2021-04-17
- [x] rtx: read rad file data
2021-04-22 19:42:19 +02:00
## 2021-04-19
- [x] rtx: light intensity-based light clusters visibility
- [x] rtx: check multiple variants of texture name (wad and non-wad)
- [x] rtx: rad liquids/xeno/... textures
2021-04-22 19:45:38 +02:00
## 2021-04-22
- [x] rtx: fix backlight glitch
- [x] rtx: textures
2021-05-01 20:48:24 +02:00
## 2021-04-24, E86
- [x] rtx: restore studio models
## 2021-05-01, E89
- [x] make a wrapper for descriptor sets/layouts
2021-05-08 23:34:42 +02:00
## 2021-05-03, E90
- [x] make map/frame lifetime aware allocator and use it everywhere: render, rtx buffers, etc
2021-05-10 18:17:03 +02:00
## 2021-05-08, E92
- [x] rtx: weird purple bbox-like glitches on dynamic geometry (tlas vs blas memory corruption/aliasing)
2021-05-17 19:57:16 +02:00
- [x] rtx: some studio models have glitchy geometry
## 2021-05-10, E93
- [x] rtx: don't recreate tlas each frame
- [x] rtx: dynamic models AS caching
## 2021-05-..-17, E93, E94
- [x] rtx: improve AS lifetime/management; i.e. pre-cache them, etc
- [x] add debug names to all of the buffers
2021-05-24 20:14:03 +02:00
## 2021-05-22, E97
- [x] add nvidia aftermath sdk
# 2021-05-24, E98
- [x] rtx: simplify AS tracking
2021-05-28 19:27:16 +02:00
## 2021-05-26, E99
- [x] rtx: fix device lost after map load
## 2021-05-28, E100
- [x] rtx: build acceleration structures in a single queue/cmdbuf
2021-06-12 21:54:53 +02:00
## 2021-06-05, E103
- [x] rtx: dynamic surface lights / dynamic light clusters
- [x] rtx: animated textures
- [x] rtx: attenuate surface lights by normal
## 2021-06-07, E104..
- [x] fix CI for vulkan branch
# 2021-06-09..12, E105..106
- [x] c3a2a: no water surfaces in vk (transparent in gl: *45,*24,*19-21)
- [x] water surfaces
2021-06-19 19:51:21 +02:00
# 2021-06-14, E107
- [x] rtx: optimize water normals. now they're very slow because we R/W gpu mem? yes
- [x] cull bottom water surfaces (they're PLANE_Z looking down)
- [x] fix water normals
2021-07-17 21:37:12 +02:00
## 2021-06-23, E109
- [x] rtx: ray tracing shaders specialization, e.g. for light clusters constants
- [x] rtx: restore dynamic stuff like particles, beams, etc
- [x] rtx: c3a1b: assert model->size >= build_size.accelerationStructureSize failed at vk_rtx.c:347
2021-07-31 21:43:35 +02:00
## 2021-07-17, E110..120
- [x] rtx: ray tracing pipeline
- [x] rtx: fix rendering on AMD
- [x] rtx: split models into a separate module
- [x] rtx: alpha test
## 2021-07-31, E121
2021-08-08 03:06:48 +02:00
- [x] rtx: alpha blending -- did a PoC
## 2021-08-02..04, E122-123
- [x] mipmaps
2021-08-22 19:44:47 +02:00
- [x] rtx: better random
2021-08-11 20:43:33 +02:00
## 2021-08-07, E124
- [x] anisotropic texture sampling
- [x] studio model lighting prep
- [x] copy over R_LightVec from GL renderer
- [x] add per-vertex color attribute
- [x] support per-vertex colors
- [x] disable lightmaps, or use white texture for it instead
## 2021-08-11, E125
2021-08-15 20:02:57 +02:00
- [x] simplify buffer api: do alloc+lock as a single op
2021-08-22 19:44:47 +02:00
## 2021-08-15, E126
- [x] restore render debug labels
- [x] restore draw call concatenation; brush geoms are generated in a way that makes concatenating them impossible
## 2021-08-16, E127
- [x] better device enumeration
## 2021-08-18, E128
- [x] rtx: fix maxVertex for brushes
2021-09-01 19:37:30 +02:00
## 2021-08-22, E129
- [x] fix depth test for glow render mode
- [x] screenshots
## 2021-08-26, E131
- [x] rtx: material flags for kusochki
2021-09-04 21:36:30 +02:00
## 2021-09-01, E132
- [x] rtx: ingest brdfs from ray tracing gems 2
- [x] rtx: directly select a triangle for light sampling
2021-09-06 23:14:59 +02:00
## 2021-09-04, E133
- [x] rtx: different sbts for opaque and alpha mask
- [x] include common headers with struct definitions from both shaders and c code
## 2021-09-06, E134
- [x] rtx: pass alpha for transparency
- [x] rtx: remove additive/refractive flags in favor or probability of ray continuing further instead of bouncing off
- [x] make a list of all possible materials, categorize them and figure out what to do
2021-10-24 18:58:41 +02:00
# E149
- [x] rtx: remove sun
- [x] rtx: point lights:
- [x] static lights
- [x] intensity "fix"
- [x] dlights
- [ ] elights
- [x] intensity fix for d/elights?
- [x] point light clusters
- [x] bsp:
- [x] leaf culling
- [x] pvs
2021-10-26 19:40:02 +02:00
- [x] rtx: better light culling: normal, bsp visibility, (~light volumes and intensity, sort by intensity, etc~)
- [x] rtx: cluster dlights
## 2021-10-24 E155
- [x] rtx: static lights
- [x] point lights
- [x] surface lights
## 2021-10-26 E156
- [x] enable entity-parsed lights by lightstyles
2021-12-22 06:37:36 +01:00
## 2021-12-21 DONE SOMEWHEN
- [x] rtx: dynamic rtx/non-rtx switching breaks dynamic models (haven't seen this in a while)
- [x] run under asan
- [x] rtx: map name to rad files mapping
- [x] rtx: live rad file reloading (or other solution for tuning lights)
- [x] rtx: move entity parsing to its own module
- [x] rtx: configuration that includes texture name -> pbr params mapping, etc. Global, per-map, ...
- [x] rtx: simple convolution denoise (bilateral?)
- [x] rtx: cull light sources (dlights and light textures) using bsp
- [-] crash in PM_RecursiveHullCheck. havent seen this in a while
2021-12-22 06:40:11 +01:00
- [x] rtx: remove lbsp
2022-09-30 10:05:07 +02:00
## 2022-09-17 E207 Parallel frames
- [x] allocate for N frames:
- [x] geometries
- [x] rt models
- [x] kusochki
- [x] same ring buffer alloc as for geometries
- [x] extract as a unit
- [x] tlas geom --//--
- [-] lights
- [x] make metadata buffer in lights
- [-] join lights grid+meta into a single buffer => pipeline loading issues
- [x] put lights data into a cpu-side vk buffer
- [-] sync+barrier upload => TOO BIG AND TOO SLOW, need to e.g. track dirty regions, compactify stuff (many clusters are the same), etc
- [x] scratch buffer:
- should be fine (assuming intra-cmdbuf sync), contents lifetime is single frame only
- [x] accels_buffer:
- ~~[ ] lifetime: multiple frames; dynamic: some b/tlases get rebuilt every frame~~
- ~~[ ] opt 1: double buffering~~
- [x] opt 2: intra-cmdbuf sync (can't write unless previous frame is done)
- [x] uniform_buffer:
- lifetime: single frame
- [x] tlas_geom_buffer:
- similar to scratch_buffer
- BUT: filled on CPU, so it's not properly synchronsized
- fix: upload using staging?
- [x] double/ring buffering
