Map in Go is a very common data structure that allows us to quickly store and retrieve key-value pairs. However, when using map in concurrent scenarios, there are still some issues that need to be paid attention to.
This article will explore whether maps in Go are concurrency-safe and provide three solutions to solve concurrency problems.
Let's answer the question first, the answer isConcurrency is not safe。
Looking at a code example, what happens when two goroutines write to the same map at the same time?
package main
import "sync"
func main() {
m := make(map[string]int)
m["foo"] = 1
var wg
(2)
go func() {
for i := 0; i < 1000; i++ {
m["foo"]++
}
()
}()
go func() {
for i := 0; i < 1000; i++ {
m["foo"]++
}
()
}()
()
}
In this example, we can see that two goroutines will try to write to the map at the same time. When running this program, we will see an error:
fatal error: concurrent map writes
In other words, in concurrent scenarios, this way of operating maps is not possible.
Why is it unsafe
Because itNo built-in lock mechanismTo protect multiple goroutines to read and write operations at the same time.
When multiple goroutines read and write operations on the same map at the same time, data competition and inconsistency will occur.
As in the example above, when two goroutines try to update the same key-value pair at the same time, the final result may depend on which goroutine completes the update operation first. This uncertainty can cause the program to error or crash.
The Go language team did not design maps to be concurrently secure because it would increase program overhead and reduce performance.
If the map has a lock mechanism built-in, locking and unlocking operations are required every time the map is accessed, which will increase the program's run time and reduce performance.
In addition, not all programs need to use maps in concurrent scenarios, so building the lock mechanism into maps will cause unnecessary overhead for programs that do not require concurrent security.
In actual use, developers can choose whether the concurrent security of maps needs to be ensured according to the needs of the program, thereby making a trade-off between performance and security.
How to be safe in concurrency
Next, we introduce three concurrency security methods:
- Read and write lock
- Sharding and locking
Add read and write lock
The first method is to useRead and write lock, this is the easiest way to think of. Add a read lock during a read operation and a write lock during a write operation.
package main
import (
"fmt"
"sync"
)
type SafeMap struct {
Map map[string]string
}
func NewSafeMap() *SafeMap {
sm := new(SafeMap)
= make(map[string]string)
return sm
}
func (sm *SafeMap) ReadMap(key string) string {
()
value := [key]
()
return value
}
func (sm *SafeMap) WriteMap(key string, value string) {
()
[key] = value
()
}
func main() {
safeMap := NewSafeMap()
var wg
// Start multiple goroutines for writing operations for i := 0; i < 10; i++ {
(1)
go func(i int) {
defer ()
(("name%d", i), ("John%d", i))
}(i)
}
()
// Start multiple goroutines for reading operations for i := 0; i < 10; i++ {
(1)
go func(i int) {
defer ()
((("name%d", i)))
}(i)
}
()
}
In this example, we define aSafeMapStructure, it contains aAnd onemap[string]string。
Two methods are defined:ReadMapandWriteMap. existReadMapIn the method, we use a read lock to protect the read operation of map. existWriteMapIn the method, we use a write lock to protect the write operation to map.
existmainIn the function, we initiate multiple goroutines to perform read and write operations, and these operations are all safe.
Sharding and locking
In the above example, the requirements are achieved by locking the entire map, but relatively speaking, the lock will greatly reduce the performance of the program. How to optimize it? One of the optimization ideas is to reduce the granularity of the lock and not lock the entire map.
This method isSharding and locking, divide this map into n blocks, and the read and write operations between each block do not interfere with each other, thereby reducing the possibility of conflict.
package main
import (
"fmt"
"sync"
)
const N = 16
type SafeMap struct {
maps [N]map[string]string
locks [N]
}
func NewSafeMap() *SafeMap {
sm := new(SafeMap)
for i := 0; i < N; i++ {
[i] = make(map[string]string)
}
return sm
}
func (sm *SafeMap) ReadMap(key string) string {
index := hash(key) % N
[index].RLock()
value := [index][key]
[index].RUnlock()
return value
}
func (sm *SafeMap) WriteMap(key string, value string) {
index := hash(key) % N
[index].Lock()
[index][key] = value
[index].Unlock()
}
func hash(s string) int {
h := 0
for i := 0; i < len(s); i++ {
h = 31*h + int(s[i])
}
return h
}
func main() {
safeMap := NewSafeMap()
var wg
// Start multiple goroutines for writing operations for i := 0; i < 10; i++ {
(1)
go func(i int) {
defer ()
(("name%d", i), ("John%d", i))
}(i)
}
()
// Start multiple goroutines for reading operations for i := 0; i < 10; i++ {
(1)
go func(i int) {
defer ()
((("name%d", i)))
}(i)
}
()
}
In this example, we define aSafeMapStructure, which contains a length ofNThe map array and one length ofNThe lock array.
Two methods are defined:ReadMapandWriteMap. In both methods, we use onehashFunctions to calculatekeyWhich map should be stored in. Then read and write the map.
existmainIn the function, we initiate multiple goroutines to perform read and write operations, and these operations are all safe.
There is an open source project orcaman/concurrent-map that is done through this idea. Interested students can take a look.
Finally, in the built-insync packageThere is also a thread-safe map in China (Go 1.9+), which achieves performance improvements in certain specific scenarios by separating read and write.
package main
import (
"fmt"
"sync"
)
func main() {
var m
var wg
// Start multiple goroutines for writing operations for i := 0; i < 10; i++ {
(1)
go func(i int) {
defer ()
(("name%d", i), ("John%d", i))
}(i)
}
()
// Start multiple goroutines for reading operations for i := 0; i < 10; i++ {
(1)
go func(i int) {
defer ()
v, _ := (("name%d", i))
(v.(string))
}(i)
}
()
}
With official support, the code has become much less instantly and is much more convenient to use.
In this example, we use the built-inTypes to store key-value pairs, useStoreMethod to store key-value pairs, usingLoadMethod to get key-value pairs.
existmainIn the function, we initiate multiple goroutines to perform read and write operations, and these operations are all safe.
Summarize
Maps in Go are not concurrently secure.
Concurrent insecure may occur when multiple goroutines access the same map at the same time. This is because maps in Go do not have built-in locks to protect access to maps.
Nevertheless, we can still use some methods to achieve concurrent security of maps.
One method is to use a read-write lock, add a read-lock during a read operation and a write-lock during a write operation.
Another method is to shard lock, divide the map into n blocks, and read and write operations between each block do not interfere with each other, thereby reducing the possibility of conflict.
In addition, there is also a thread-safe map in the built-in sync package (Go 1.9+), which achieves performance improvements in certain specific scenarios by separating read and write.
This is the article about in-depth discussion on whether maps in Go are concurrent security and solutions. For more related Go map content, please search for my previous articles or continue browsing the related articles below. I hope everyone will support me in the future!