Update hash staff & engine sync #303

Wend4r · 2025-03-11T06:27:55Z

Preparing changes before Source 2 Engine sync (HLX branch) into Counter-Strike 2. The methods/definitions were added carefully for compatibility with the current game code (so far).

Sync HL2SDK (CS2)

`CBufferStringN`

Perform an one-time Purge of CBufferStringN.
Set the template default size to 128 bytes (most common use case).

`CUtlStringToken`

Added constexpr constructor to inline the hash value.
Added operators for comparison in the utl containers.
Added CUtlStringTokenHashMethod class to pass the compare function into CUtlHash.
Expanded MakeStringToken capabilities.
Added MakeStringToken2 function to calculate a runtime hash with CBufferStringN.

`CUtlSymbolTable`

Added FOR_EACH_SYMBOL(_BACK) macro to iterate by the table.
Added Hash methods to get a hash by the sybmol.
Added Remove method to remove the symbol (is needed to remove the pluginself game system factory for safe unloading).
Added definite hash table methods.

`CUtlStringMap`

Added FOR_EACH_STRING_MAP macro to iterate by the map.
Added FindAndRemove method (for gsplugin's safe unloading)

`CUtlSymbolTableLargeBase`

Added FOR_EACH_SYMBOL_LARGE(_BACK) macro to iterate by the table.
Made String, Hash & GetNumString methods public to get the symbol large data.

`CKV3MemberName`

Added CUtlStringToken inherit for the child methods & compare operators.
Added constexpr constructor to inline the hash value & string.
Added Make method & constructors to calculate a runtime hash.

KeyValues3

Added FOR_EACH_KV3_ARRAY(_BACK) macro to iterate by the kv3array.
Added FOR_EACH_KV3_TABLE(_BACK) macro to iterate by the kv3table.
Added CKV3MemberNameWithStorage container (Pulse thing)
Added CKV3MemberNameSet definition (Pulse thing; aka KeyValues3 with KV_TYPE_ARRAY)
Added CKeyValues3StringAndHash definition (old CKV3MemberName name).
Added KeyValues3LowercaseHash_t definition (aka CUtlStringToken).
Added IsArray/GetArray method.
Added IsTable/GetTable method.
Added Internal_FindMember method (also in the kv3table).
Renamed GetMemberNameEx to GetKV3MemberName (like-in the engine).
Added HasInvalidMemberNames & SetHasInvalidMemberNames methods (also in/of the kv3table; used for unserialize an ekv).
Added RenameMember method.
Added OverlayKeysFrom method.
Expanded PrepareForType arguments to pass ones into Alloc.

Changes that will affect the current SDK's works

Add a new integer template argument to the base utl containers (by default 0) that usually uses together store Vulkan or animgraph containers (@gishi523 or more engine research; not to be confused with non-added CUtlVectorBase<T, I, A>; important for the named signatures of platform-overloaded functions)
Delete the schema static fields & add a baked unbinding field funciton (done locally, preparing)
Update ICvar & convar structures (in the dota branch)

Update `CUtlSymbolLarge` container

GAMMACASE

Overall thanks for the work, highly appreciate it!

public/tier1/UtlStringMap.h

GAMMACASE · 2025-03-13T13:39:57Z

public/tier1/UtlStringMap.h

+	CUtlSymbol Insert( const char *pString, const T &item )
+	{
+		CUtlSymbol symbol = m_SymbolTable.AddString( pString ); // implicit coercion
+		if ( m_Vector.Count() > symbol ) 
+		{
+			// this string is already in the dictionary.
+
+		}
+		else if ( m_Vector.Count() == symbol )
+		{
+			// this is the expected case when we've added one more to the tail.
+			m_Vector.AddToTail( item );
+		}
+		else // ( m_Vector.Count() < symbol )
+		{
+			// this is a strange shouldn't-happen case.
+			AssertMsg( false, "CUtlStringMap insert unexpected entries." );
+			m_Vector.EnsureCount( symbol + 1 );
+			m_Vector[symbol] = item;
+		}
+		return symbol;
+	}


There's no real need in such a method as utlstringmap expects you to do insertions directly by a key, example:

stringmap[newkey] = newvalue;

This is needed to add a new string key & insert it into the expanded utlvector.
CUtlStringMap's operators are desiged for getting/setting an existing key.

They are tho, how else would you expect new entries to be added there? You can test yourself these with adding new keys. This code in particular handles that

hl2sdk/public/tier1/UtlStringMap.h

Lines 30 to 39 in c6ac506

T& operator[]( const char *pString )

{

CUtlSymbol symbol = m_SymbolTable.AddString( pString );

int index = ( int )symbol;

if( m_Vector.Count() <= index )

{

m_Vector.EnsureCount( index + 1 );

}

return m_Vector[index];

}

GAMMACASE · 2025-03-13T13:42:19Z

public/tier1/utlsymbol.h

+	// Remove once symbol element.
+	// @Wend4r: The table is not designed for that.
+	void Remove( CUtlSymbol id ) { m_HashTable.Remove( id ); m_MemBlocks[ id ] = MEMBLOCKHANDLE_INVALID; }


As per your note, container isn't designed for such removals, so no need to hack them in and potentially hurting other code relying on such behavior, please, revert this.

This is necessary for gsplugins to remove game system factory from &IGameSystem::sm_GameSystemFactories for safe unloading a meta plugin & map changing. Are you sure?

If all you need is to remove entry from a stringmap, just empty it like you did in the method but do that externally, there's no need for such methods just to satisfy that particular feature.

Okay, I'll think about it, how to do it without the Remove method

Well since all you do in a stringmap::Remove method is vec[symbol] = {}, you can achieve the same with stringmap[key] = {}

tier1/keyvalues3.cpp

GAMMACASE · 2025-03-13T16:37:09Z

tier1/keyvalues3.cpp

@@ -1758,6 +1837,40 @@ void CKeyValues3Table::CopyFrom( KeyValues3 *parent, const CKeyValues3Table* src
 		EnableFastSearch();
 }

+void CKeyValues3Table::RenameMember( KeyValues3 *parent, KV3MemberId_t id, const CKV3MemberName &newName )


While you at it, how about adding an assert to ensure the id is within valid range, for this method and any other that take id (like RemoveMember)

GAMMACASE · 2025-03-13T16:44:54Z

public/tier1/utlstringtoken.h

+#include <string.h>
+#include "bufferstring.h"


These includes introduce cyclic include dependency issues which causes some parts of the code to fail. As per the comments below these are better removed from here anyway!

Solved it in the slightly different way

I don't think what you did is sane enough, moving stuff around like that to satisfy the needs isn't ideal especially for stuff that always been there, please, just remove the utlstring and bufferstring from stringtoken as that will fix every issue related to this.

Co-authored-by: GAMMACASE <31375974+GAMMACASE@users.noreply.github.com>

Co-authored-by: Co-authored-by: GAMMACASE <31375974+GAMMACASE@users.noreply.github.com>

Co-authored-by: GAMMACASE <31375974+GAMMACASE@users.noreply.github.com>

Use `DefaultHashFunctor< CUtlStringToken >` instead Co-authored-by: GAMMACASE <31375974+GAMMACASE@users.noreply.github.com>

Co-authored-by: GAMMACASE <31375974+GAMMACASE@users.noreply.github.com>

…alue` back Update hash value type Co-authored-by: GAMMACASE <31375974+GAMMACASE@users.noreply.github.com>

Co-authored-by: GAMMACASE <31375974+GAMMACASE@users.noreply.github.com>

To prevent non-kv3(array/table) values into a loop body

Wend4r · 2025-03-15T07:49:44Z

It works fine with CS2Fixes-based plugin (on SteamRT side), but I'll continue to work on this PR

tier1/keyvalues3.cpp

public/tier0/fasttimer.h

Wend4r and others added 9 commits March 11, 2025 05:09

Add CUtlStringMap::FindAndRemove method

80aff5e

Removed Plat_FatalErrorFunc from deadlock

292e03c

CUtlStringMap: remove hash element of symbol table

d8c5f54

Update CUtlSymbolTable & hash staff

caa5928

Add CCommandBuffer::SplitCommands method

bcb8069

Locate CKeyValues3Table::m_bHasInvalidMemberNames field

c4e277d

Add KeyValues3::RenameMember method

a4982c5

Update `CUtlSymbolLarge` container

Add KeyValues3::OverlayKeysFrom method

8441bc8

Resolve UtlSymLargeId_t differences

c806485

Wend4r marked this pull request as draft March 11, 2025 06:28

Resolve UtlSymLargeId_t differences (2)

c6ac506

GAMMACASE added the source2 label Mar 11, 2025

GAMMACASE requested changes Mar 13, 2025

View reviewed changes

Wend4r and others added 17 commits March 15, 2025 08:07

Update CUtlStringMap constructor

998f037

Co-authored-by: GAMMACASE <31375974+GAMMACASE@users.noreply.github.com>

Update CUtlStringMap access operators

49b6339

Co-authored-by: Co-authored-by: GAMMACASE <31375974+GAMMACASE@users.noreply.github.com>

Uncomment CUtlStringMap::Count's assert

dfc32eb

Co-authored-by: GAMMACASE <31375974+GAMMACASE@users.noreply.github.com>

Remove CUtlStringMap::First's method

ae85a4e

Co-authored-by: GAMMACASE <31375974+GAMMACASE@users.noreply.github.com>

CBufferStringN: replace uintp with size_t

6c97622

Co-authored-by: GAMMACASE <31375974+GAMMACASE@users.noreply.github.com>

Update FOR_EACH_KV3_ARRAY(_BACK) & FOR_EACH_KV3_TABLE macros

2b2c447

Co-authored-by: GAMMACASE <31375974+GAMMACASE@users.noreply.github.com>

Add CKV3MemberHash definition

1c5f015

Co-authored-by: GAMMACASE <31375974+GAMMACASE@users.noreply.github.com>

Remove CUtlStringTokenHashMethod class

3a6ff85

Use `DefaultHashFunctor< CUtlStringToken >` instead Co-authored-by: GAMMACASE <31375974+GAMMACASE@users.noreply.github.com>

Update CUtlStringToken's constructors

bd0c6b3

Co-authored-by: GAMMACASE <31375974+GAMMACASE@users.noreply.github.com>

Remove CUtlSymbol interate opertators

892434b

Co-authored-by: GAMMACASE <31375974+GAMMACASE@users.noreply.github.com>

Update UtlSymId_t operator method

44e3ddf

Co-authored-by: GAMMACASE <31375974+GAMMACASE@users.noreply.github.com>

Update CUtlSymbolTableLargeBaseTreeEntry_t structure

eaf586e

CUtlSymbolTableLargeBaseTreeEntry_t: Rename Hash method to `HashV…

f6ca035

…alue` back Update hash value type Co-authored-by: GAMMACASE <31375974+GAMMACASE@users.noreply.github.com>

Rename CUtlSymbol::Hash to CUtlSymbol::MakeHash

37a2d4b

Co-authored-by: GAMMACASE <31375974+GAMMACASE@users.noreply.github.com>

Format kv3 source & make more readable cluster things

5f9f05b

Correct kv3(array/table) getting ones

cfcd835

HL2SDK: resolve include conflicts & fix errors

c1ee083

HL2SDK: fix FOR_EACH_KV3 macro

145f2f9

To prevent non-kv3(array/table) values into a loop body

GAMMACASE reviewed Mar 15, 2025

View reviewed changes

tier1/keyvalues3.cpp Show resolved Hide resolved

Wend4r added 2 commits March 15, 2025 12:26

Fix kv3table & cluster issues

111a6bf

Fix KeyValues3::GetCluster method

936f277

GAMMACASE reviewed Apr 1, 2025

View reviewed changes

public/tier0/fasttimer.h Outdated Show resolved Hide resolved

Merge branch 'cs2' into hl2sdk-cs2

5e3b264

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Update hash staff & engine sync #303

Update hash staff & engine sync #303

Wend4r commented Mar 11, 2025 •

edited

Loading

GAMMACASE left a comment

GAMMACASE Mar 13, 2025

Wend4r Mar 15, 2025

GAMMACASE Mar 15, 2025

GAMMACASE Mar 13, 2025

Wend4r Mar 15, 2025

GAMMACASE Mar 15, 2025

Wend4r Mar 15, 2025

GAMMACASE Mar 15, 2025 •

edited

Loading

GAMMACASE Mar 13, 2025

GAMMACASE Mar 13, 2025

Wend4r Mar 15, 2025

GAMMACASE Mar 15, 2025

Wend4r commented Mar 15, 2025

	T& operator[]( const char *pString )
	{
	CUtlSymbol symbol = m_SymbolTable.AddString( pString );
	int index = ( int )symbol;
	if( m_Vector.Count() <= index )
	{
	m_Vector.EnsureCount( index + 1 );
	}
	return m_Vector[index];
	}

		#include <string.h>
		#include "bufferstring.h"

Update hash staff & engine sync #303

Are you sure you want to change the base?

Update hash staff & engine sync #303

Conversation

Wend4r commented Mar 11, 2025 • edited Loading

Sync HL2SDK (CS2)

CBufferStringN

CUtlStringToken

CUtlSymbolTable

CUtlStringMap

CUtlSymbolTableLargeBase

CKV3MemberName

KeyValues3

Changes that will affect the current SDK's works

GAMMACASE left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

GAMMACASE Mar 15, 2025 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Wend4r commented Mar 15, 2025

Wend4r commented Mar 11, 2025 •

edited

Loading

`CBufferStringN`

`CUtlStringToken`

`CUtlSymbolTable`

`CUtlStringMap`

`CUtlSymbolTableLargeBase`

`CKV3MemberName`

GAMMACASE Mar 15, 2025 •

edited

Loading