golang uses map to support high concurrency (10 million operations 14ms)

There are two problems with the native map of the language:

1) Not thread-safe;

2) When the data volume is large, you need to avoid using string, etc., as GC is under great pressure;

Someone implemented the related cocurent-map using generics, (/orcaman/concurrent-map) However, the key value part still uses string by default. In order to improve efficiency, some modifications have been made to enable the key value to be customized as well:/robinfoxnan/go_concurrent_map

Basic usage method:

	// Create a new map.
	m := [uint64, string]()
 
	// Sets item within map, sets "bar" under key "foo"
	(199010212, "bar")
 
	// Retrieve item from map.
	bar, ok := (199010212)
	(bar, ok)
 
	// Removes item under key "foo"
	(199010212)

In order to implement counters, etc., it is necessary to update during locking, and a callback function is required:

// Countertype BaseCounter struct {
	Count     uint64
	CountLast uint64
}
 
var MapOfAppUserCount ConcurrentMap[uint64, *AppUserCounter]
 
func InitMaps() {
	MapOfAppVistedCount = NewConcurrentMap[uint64, *BaseCounter]()
}
 
// If there is no value, set; if there is, update; the newly added part is passed through the new value!func appAddCallBack(exist bool, valueInMap *BaseCounter, newValue *BaseCounter) *BaseCounter {
	if exist == false {
		return newValue
	} else {
		 += 
		return valueInMap
	}
}
 
// Add i to the application counterfunc AppAddBy(key uint64, i uint64) uint64 {
	c := BaseCounter{i, i}
	res := (key, &amp;c, appAddCallBack)
	if res != nil {
		return 
	}
	return 0
}

The counter is used as follows:

()
(i, 1)

performance:

1) Single thread initializes a counter of 1~1000w, 2412 ms

2) Distribute to 100 coroutines, 14ms

The test code is as follows:

 func testSingle() {
	()
	timeUnixNano1 := ().UnixMilli()
	// 1 million updates	for i := uint64(0); i &lt; 10000000; i++ {
		(i, 1)
	}
	timeUnixNano2 := ().UnixMilli()
	delta := timeUnixNano2 - timeUnixNano1
	("cost: ", delta, " ms")
 
	count := (1, 1)
	(count)
	count = (1, 2)
	(count)
	count = (1, 3)
	(count)
}
 
var N int = 10000000
 
func doInsert(n int, index int, g *) {
	m := N / n
	start := index * m
 
	//("thread ", index, "from ", start)
	for i := uint64(start); i &lt; uint64(m); i++ {
		(i, 1)
	}
	if g != nil {
		()
	}
}
 
func testMulti() {
	()
 
	group := {}
	n := 100
	(n)
	timeUnixNano1 := ().UnixMilli()
	for i := 0; i &lt; n; i++ {
		go doInsert(n, i, &amp;group)
	}
	()
	timeUnixNano2 := ().UnixMilli()
	delta := timeUnixNano2 - timeUnixNano1
	("cost: ", delta, " ms")
 
}

This is the article about golang making map support high concurrency (10 million operations 14ms). For more related golang map concurrency content, please search for my previous articles or continue browsing the related articles below. I hope everyone will support me in the future!