Skip to content
IT Docs

GO

Possible Architecture

  • layering
    • handlers - external interface for http, rpc, message broker, that converts internal models to external
    • service - contains logic and operate with internal models with a help of repo & external service
    • external service - can be directly RPC service OR wrapped with custom interface
    • repo - contains logic to retrieve data from DB by executing SQL (or similar) queries
      • basically a sub-type of external service
      • might or might not have own models and errors
  • everything is passed through DI for testability
  • commonly tests are done on each component as units, repo tests with lightweight DB AND functional tests when you spin-up your app and execute actions with it through public interface
    • e2e is also a possibility, when you aren’t mocking external services, but using them
  • context is stored as global var that populated on start
  • logs can be written to VL or similar solution
  • graphana+prometheus can be used for alerts and metrics

Roadmap

  • intro
    • great lang for BE services, CLI tools and distributed systems with GC, strong types, compilation (to single binary) and strong std, that features performance, great error handling and concurrency management
    • simplistic & easy to learn
    • go command features:
      • mod - depth management
      • build - compilation
      • run - dev run
      • test - testing
      • fmt - formatting
      • etc
  • syntax
    • vars:
      • changeable: var OR := shorthand
        • var declares AND allows to avoid initialization & assignment
          • if value is not provided, go will assign default values:
            • 0, "", false OR nil (pointers, slices, maps)
        • := always must have value, infers type, can be used inside fn only
      • unchangeable: const
        • you can use itoa keyword to create int sequence starting from 0 (will start from zero per const block)
      • can have explicit type OR infer it
      • go has concept of shadowing AND scoping (block, fn, package)
      • data types:
        • int8-64, uint8-64, float32/64, complex numbers, boolean, string, bytes (alias for uint8), rune (alias for int32)
          • usually you should use just int, which is often int32 OR 64, based on architecture
          • go strict with types and require conversion
          • for bools go have: &&, !, ||, ==, != logical operations and checks
        • statically typed and will error at compile time
      • composite data types:
        • array - fixed size sequence of sam-type elements
          • size is part of type
          • initialize with zero values
          • passed as value
        • slice - more robust version of array, that supports dynamic sizing, append and copy operations
          • each slice has capacity (can be pre-allocated: make([]T, length, capacity); for performance), that determines when memory re-allocation occurs
            • make also creates maps make(map[string]int) and channels make(chan int)
            • you can convert slice to array [N]T(slice) (will panic if slice has lower then N elements)
            • if you need to operate on array memory as a slice you can convert it: array[:] OR array[start:end]
        • map - key/value data type, that supports data storage (with keys of comparable types)
          • except basic map operations you can idiomatically check for existence value, ok := map[key]
            • comma-ok idiom used to distinguish presence in map, even in case if value is default
            • same idiom can be used to check type assertion without panic: value, ok := interface.(Type) AND if channel closed: values, ok <-ch
        • string - immutable (manipulations will copy string) sequence of bytes (always encoded as UTF-8)
          • to effectively iterate over UTF-8 strings you can use rune, that will represent UTF-8 char
          • note that len prints number of bytes in str, not runes
          • rune can also be created via ''
          • “ can be used to define strings without escape sequences and WITH formatting preservation
          • "" define regular strings with escape sequences
          • note:
            • for range will iterate string by runes, BUT index access will return bytes
        • struct - used to define interface for object with methods only
          • functions attached separately
          • struct support tags, that define how to serialize data into struct, ex: json:"name,omitempty"
            • omitempty wont add field to JSON if it has zero value
            • json:"-" will prevent (un)marshal
            • note that not exported fields are ignored in (un)marshal process
          • struct can be embedded with other struct, enabling composition pattern

type Address struct { Street, City, State string }

type Person struct { Name string Address }

p := Person{ Name: “Alice”, Address: Address{“123 Main St”, “Anytown”, “CA”}, } fmt.Println(p.Street) ``` - conditions: - if and if-else - if supports initialization in condition: if num := 9; num < 0 {...} - switch - supports: multiple values per case, type switch, expression & fallthrough (need to define by fallthrough keyword) - loops - for is basic and single loop in go, supports condition, post statement, initialization, continue and break with labels for nested loops - for initialisation; condition; post { ... } - for range iterator style syntax that can be used for arrays, slices, maps, strings, and channels - returns index/key (except channel), value for all types - _ syntax is used to omit value - notes: - map: - can’t be modified on iteration (except deletion) - iterates in random order - string: - iterates by runes - goto can be used to move control flow to chosen label within fn - docs - go has Godoc for comments as docs - you can add comment before fn to act as inline doc AND at start of file to act as doc for whole file - supports markup - can be parsed by Godoc package to form HTML view of docs - you can write specific types of tests, that will act as example, by using specific file name example_{FUNCTIONALITY_NAME}_test.go AND fn name (starts with Example) - functions - first class citizens (can be assigned to var, passed as param) - has multiple returns (funct name () (Type1, Type2, commonly used for error handling), named returns (funct name () (res1 int, res1 int) { res1 = 1; res2 = 2; return; }, allows for variadic parameters (of same type, func name (args ...Type), treated as slice inside fn) - can be anonymous (func(){}), which are common for lambdas OR as var - functions create closures - all variables accepted by value, except pointers, slices and maps (pointer itself is still copied, BUT memory, that it points to, isn’t) - pointers - declared as *Type, pointer of value obtained via &Var, value is read or assigned via *p OR *p = smth - for structs: (*p).field OR p.field shorthand - can’t do arithmetic - memory is managed via GC, BUT it is still important to omit copying large amount of data, leaking memory with redundant allocations - GC runs concurrently with program and tries to avoid large resource consumption - methods - similar to functions, BUT act as methods on defined types and can have pointer/value receiver - pointer receiver is used for large structs OR when state modification needed func (p *Type) name () {} - value receivers can also be used, even if called on pointer (dereferencing will be done under the hood) funt (v Type) name () {} - interface - used to define type without implementation - interface{} accepts any type - commonly used for generics before generics - used to handle unknown data - to use anything we can narrow type by: - assertions - value.(Type) with panic OR more safer value, ok := value.(Type) - type switches - switch v := i.(type) { ... } - type Name interface { fn1 () ReturnType} - can be embedded for composition (same as structs) - generics - introduces reusability without sacrificing performance OR strong types - func name[T ConstrainType] (s T) T - for any you can use interface{} OR any - can be called like name[int] (...) OR generic can be inferred - structs can be generic too type Struct[T any] struct {} - you can build own constrains: - type Number interface { int | int8 | float32 } - type Number {int Calc () int} - type Number interface { ~int | ~int8} to allow accepting types, who has int or int8 underlying type - (they can be inlined, using same syntax) - there is constraints package that have expected Float, Integer ect constraints and more specific: comparable, ordered - errors - errors are values that need to be explicitly handled - created via fmt.Errorf() OR errors.New() - errors.New great for static error messages OR predefined constant errors (Sentinel Errors) - fmt.Errorf great for formatted errors, with possibility to pass other error via %w to wrap it inside, while preserving context - errors.Is(wrapped, base) OR errors.As(wrapped, baseType) can be used to work with wrapped errors - except default %s, %d, %f you have: - %+v for key:val structs - %T for type - %q for quoted string - always returned as last value, defaults to nil and handled as if err != nil - built-in error interface has single Error() string method that returns text representation of error - for exceptional cases you have panic() that will unwind the stack and kill app - to prevent app death you have recover() in deferred functions

  • code organization
    • module (app) in go is defined by go.mod file, which includes path (often repo URL) to module & dependencies
      • go mod init creates basic module
      • go mod tidy cleans-up, adds and optimizes deps
        • go.sum checksum updated as result for reproducible builds
      • go mod vendor creates minimal required local copy of all depth
        • app will fail in case of mismatch between vendor and actual deps
      • go get installs deps
    • code is broken by packages
      • defined by first line via package name
      • exports are defined by capital letter
      • import is done as:
        • import "name"
        • import ( "name1", "name2" )
        • import localName "name"
      • rules:
        • no circular imports
        • main package for executable
        • lowercase naming
        • import path must be uniq
      • to publish you required to share module via version control system with semantic versioning
      • notes:
        • files in package can “see” each other
        • all files are compiled as single package, that can be linked to other package
        • dead code elimination is done in compilation step to omit any unreachable from main symbols
  • concurrency
    • goroutines is way to execute concurrent, multi-threaded code
      • lightweight and managed by runtime
      • communication is done via channels
      • syntax: go fn() to execute fn as goroutine
    • note that main termination will kill goroutines, it won’t await them by default, BUT we can block it via channels
    • data is shared via channels
      • make(chan, Type)
      • can be buffered (queue N entities in memory) OR unbuffered (block channel until entity in processing)
      • you can multiplex several channels via select/case statement
        • will block until any case or default is ready
        • will execute case of ready channel OR random if multiple ready
        • will execute default immediately if other are blocking
        • use-case: channels with timeouts
    • while gorouting is lightweight it might be beneficial to limit it by some amount via workerpool (allocate some fixed amount of goroutines and buffer other requests)
    • for better management you can use
      • sync, that provides: mutex, reader-writer mutex, waitgroup (wait until all goroutines finish), once (call fn once even if done from multiple routines)
      • context, that provides:
        • value passing down the stack
        • deadlines with cancelations
          • check ctx.Done() and abort
        • ---
        • often passed as first request
        • use-cases: HTTP timeouts, database deadlines, goroutine cancellation coordination
    • patterns:
      • fan-in - merge work (done by select)
      • fan-out - distribute work for faster parallel execution
      • pipeline - chain parallel working steps
    • you can use -race flag to detect unsynchronized used of shared memory by different routines
  • go has reach standard lib:
    • io to handle files and streams
      • and it’s brother bufio for buffered io operations (better performance due to reduced sys calls)
    • os for file, env, process, sys and network operations
      • cross-platform
    • flag simple util for parsing CLI flags
    • time working with time
      • has rich timezones support and handles edge-cases
    • json marshaling & unmarshaling structs into JSON
    • log for logging
      • and newer slog for logs with built-in levels, cleaner key-val interface AND json formatting
    • regexp for RegExp
      • support compiled expressions to avoid recompiling
    • go:embed to create self-contained binaries
    • testing
      • file must end with _test & function must start with Test
        • fn accepts t *testin.T that is used to validate results
      • goes encourage pattern of simple loops in tests, where you accept slice of structs to run against same setup (table-driven tests)
      • mocks & stubs for checking fn calls & stubbing response in combination with DI is used to test without dependencies
      • httptest package can be used to record and emulate network requests without network
      • if fn starts with Benchmark you can run b *testin.B to write performance tests and run them via go test -bench=.
      • go test -cover exposes coverage
    • net/http for building RESTful web servers
      • provides TLS, HTTP/2, cookies & multipart form handling
    • database/sql for executing queries against DB
      • go also supports extended libs like pgx for different DBs, ORMs & query builders
      • consider using tooling for migrations & connection pools
    • for ws and SSE you can use existing libs
  • tooling
    • standard:
      • go run execute file without building
      • go build build program as binary
        • cross OS, has custom flags, optimized by default AND creates single statically linked binary (by default)
        • artifact is easy to deploy
        • note that for conditional build (platform specific code, feature flags etc) you can utilize //go:build tagging
        • -gcflags="-m" can be used to show what vars was heap allocated and utilized GC
      • go install package@version to build and install programs system-wide
        • commonly used for CLI tooling
      • go fmt standard opinionated formatter
      • go mod init/tidy/download for dependency management
      • go test to find & run tests, examples & benchmarks
        • has coverage & parallel running
        • t.Parallel() marks test as possible to run concurrently
        • go test will run go vet to check for potential bugs (incorrect printf, dead-code, problematic struct tags, nil pointer dereferencing)
        • GOOS & GOARCH env vars control targeted OS & architecture version
      • go clean to clean package & build caches
      • go doc to extract docs of package/fn from go doc comments
      • go version
      • go generate for executing commands in //go:generate comments
        • use-cases: embedding data on build-time, meta-programing, code gen from templates
        • can be used to avoid bash scripts and inlining & running needed commands in go file
      • go tool pprof for profiling (CPU, RAM, goroutines, blocking opearions)
      • go tool trace for analyzing traces (goroutines, sys-calls, GC, scheduling)
        • can be added by runtime/trace
    • goimports - automatically remove & resolve missing imports
    • linting
      • golangci-lint - parallel runner for quality checkers
      • staticcheck, revive - configurable third-party linters
    • govulncheck - CVE finder for dependencies
  • memory management in depth
    • GOGC can be used to increase/decrease heap-size (memory usage), while decreasing/increasing CPU load
      • 100(%) is default value and state that heap can use additional 100% of currently used RAM by program itself
    • GOMEMLIMIT can used to balance high GOGC by limiting max allowed memory for program
      • soft limit to avoid program stolling itself
      • keep it 10% below actual limit of container
    • focus on actual physical memory, go holds to reserved vMemory
  • reflection std package can be used to examine & manipulate unknown data (ex: ORM, JSON Marshaling) with additional performance overhead’
  • unsafe allows for C-style memory management & pointer arithmetic
  • CGO can be used to link C and GO code via comments in both directions
    • has performance cost and breaks cross-compilation & static linking
    • non-preferred
  • GO supports dynamic imports called plugins
    • plugin must follow type interface that defines interface and called SomeNamePlugin
    • plugin must export Plugin var
    • must be built with go build -buildmode=plugin as .so file
    • in app is read from path, extracted by symbol and casted to type
    • problems: unix-only, complex, versioning problem

Effective GO

  • formatting
    • tabs over space
    • no line limit
    • reduced number of parentheses (spacing to order math operations, no need for parentheses in control structs)
  • naming
    • package is imported by name AND it’s name is accessor
      • should be short, consistent, understandable, lowercase and single word
      • should match it’s directory name
    • GetName -> Name
    • ToString -> String
    • Reader for single method interfaces
    • camelCase
  • semicolons
    • auto-inserted
    • required in control statements, as delimiter for single line expressions
  • control structs
    • support internal var initialization if err := fn(); err != nil { ... }
    • := can be used for reassignment and declaration
    • case ' ', '?', '&', '=', '#', '+', '%': is allowed
    • empty switch will fire on true
    • defer will run clean-up fn before fn returns
      • vars are evaluated when defer is inited
      • multiple defers will be executed in LIFO order
  • data
    • allocation
      • new(T) allocs zeroed memory and returns pointer to that region
        • &Struct{} == new(Struct)
      • make(cn/slice/map) initializes underlying data structure with given params, returning value itself
    • array
      • useful for memory layouting & to avoid additional RAM usage
      • foundation for slices
      • arrays are treated as values
      • type of array is bound to size
    • slices
      • richer for of array
      • treated as references
    • maps
      • treated as references
      • similar to slices, BUT represent a key-val data struct
      • key is any comparable type
      • double delete is safe
    • printing
      • Printf, Fprintf, SprintF - format string and output to stdout, passed buffer, as string
      • Print, Println - take any number of values, format by type, join by "" or "\n" into output and print
      • for maps output is sorted by key
      • custom String() on interface can be defined to specific %v formatting
  • initialization
    • constants
      • compile time
      • can have methods on them (as all other types, except pointer and interface)
    • var
      • expression can be runtime
    • flow:
      • evaluate dependency
      • evaluate vars
      • call init function defined by file
  • types
    • type conversion may produce new value (int-> float) OR temporary change value type to allow accessing methods
      • failed type conversion with ok:= will return zero value
  • blank identifier
    • works similar-ish to /dev/null in Unix
    • import _ "path" can be used to import smth to trigger side-effect
    • var _ T = (*Struct)(nil) statically check if Struct satisfies T’s interface
      • only used if you can’t do it via actual live code
  • embedding
    • interfaces can be embedded with interfaces
    • struct can be embedded with ref to other struct
    • notes:
      • shadowing will work from top to bottom (A.m will shadow embedded A.B.m)
      • having same name on same level will cause error
  • concurrency
    • due to channels each piece of data is only present in one routine at a time
    • channel is first-class citizen and can be passed via channels for two-way communication
    • runtime.NumCPU() provides number of cores for parallel execution
      • or runtime.GOMAXPROCS(0) if we wan’t to honor user requested num of cores
  • errors
    • prefix your errors for better understanding of them
    • when running long-running processes (ex: web-server), always have top-level recover to kill failing routines and preserve server
      • panic inside deferred fn will continue unwinding stack
      • re-panic will preserve stack trace

100 Mistakes

  • be aware with shadowing
  • don’t nest too much
    • early return if possible
  • don’t overuse init fns
    • problems:
      • only possible to init state in global vars
      • may introduce dependencies, that will complicate testing
      • limited error handling
    • use-cases:
      • static config initialization
  • don’t introduce redundant getter/setter
  • avoid redundant interfaces & over-abstraction
  • mostly clients should own abstractions, producer may expose minimalistic when needed
    • same applies for return arguments
  • generally avoid any
  • generics introduce complexity, so don’t overuse
  • avoid redundant type embedding
    • redundant when you want Foo.Baz() instead of Foo.Bar.Baz()
    • redundant when you leak internals this way
  • use builder OR options patter for complex structs
    • options simplifies error handling & omits dealing with empty struct for state
    • type Option func(options *options) error[]
  • have proper project organization
    • decide on folder grouping that solves your problem and stick with it
    • best practices:
      • granularity is key
      • don’t overuse packages prematurely
      • choose proper name
        • avoid generic names like util etc
      • avoid exporting too much
      • declare package with name of repo URL, if it will be hosted
      • place internals into internal because it can’t be imported by consumer
      • recommended to place command packages under cmd dir
  • be aware of package & var name collisions
  • don’t disregard docs
    • comments for exported code & packages
      • start with name
      • comment must be a sentence with punctuation
      • must describe behavior & intention, not implementation
      • should be enough info to use code without examples
    • docs for all code
      • should provide internal info & how code does things
  • use linters, formatters & analyzers
  • use longer base identifier for number & _ delimeter for better understanding of code (0->0o)
  • be aware of int overflow
    • silent at run-time
    • causes compilation errors for constants
  • remember that floating point arithmetics implies precision errors

    • When comparing two floating-point numbers, check that their difference is within an acceptable range.

    • When performing additions or subtractions, group operations with a similar order of magnitude for better accuracy.

    • To favor accuracy, if a sequence of operations requires addition, subtraction, multiplication, or division, perform the multiplication and division operations first.

  • account for slice capacity & proper initial length
  • nil slice vs empty slice
    • nil slice - no underlying array, len == 0, cap == 0, and slice == nil is true
      • var a []int
      • doesn’t have allocation
    • empty slice - usually has len == 0, cap == 0, but slice == nil is false
      • []int{}
    • behaves differently in APIs and JSON
      • when designing API choose carefully the intended behavior
  • copy will copy elements partially if slice len is smaller
  • if two slices share array, append may modify both if capacity allows for it
    • better to avoid shared arrays OR limit capacity of copy to trigger re-allocation
    • note that if one slice with shared array exists second can’t be freed
      • this especially harmful with pointers to structs
  • don’t create slices with redundant size
  • init map with number of elements if possible
  • map can’t shrink in size
    • store pointers that can be zeroed
    • re-create smaller map
  • comparison edge-cases:
    • channel -> check if both nil OR both created by same make call
    • interfaces -> check if both nil OR have dynamic types & values
    • pointers -> check if both nil OR point to same memory
    • structs & arrays -> compare val by val (only when composed of similar types)
    • > etc can also be used for lexical string comparison
    • reflect.DeepEqual can be used with slices & maps
      • critical for performance
    • alway look for standard comparison methods like bytes.Compare for performant []byte comparison
    • comparing non-comparable will panic
  • control struct edge-cases
    • range loop exposes copy of value
    • range loop evaluates copy before loop itself
    • adding element into map while doing iteration MIGHT still expose this element in current iteration
    • defer won’t execute in each loop cycle, only before fn returns
  • strings edge-cases
    • trim removes all runes from set, trimSuffix/trimPrefix remove substring
    • use strings.Builder when appending to str in loop instead of += which re-creates string over and over
      • don’t forget to pre-allocate proper buffer size via .Grow
      • can’t be used concurrently
    • avoid byte[] to string conversion if possible
    • creating sub-string from string can lead to memory leaks just like in slices
      • not that numbers correspond to bytes not runes here
  • function edge-cases
    • diff pointer & value param
      • if param non-copiable, large or must be mutated - pointer
      • map, channel & fn are always passed by val
    • when returning nil assigned to var of interface type you actually returning interface value, SO better to return just nil
    • defer params evaluated immediately
      • to reference dynamic var use pointer to it OR closure
  • error management
    • wrapping
      • wrap to add context
        • %w to preserve error
        • %v to transform error
      • wrap to change type
        • new error type is required
      • wrapped errors can be compared via errors.As
    • known errors converted to sentinel once, unknown should be wrapped under custom error type
    • log error on higher level to avoid double logging
    • mark error as ignored via _ to make action intentional (additional comment is preferable)
  • concurrency (enables parallelism, but not actually it)
    • it won’t always be faster then sequential
    • channel vs mutex
      • when we need synchronization prefer mutex, BUT when we need some form of communication prefer channel
      • channel preferable for concurrent flows, WHILE mutex for parallel
    • context:
      • has deadline - when to stop activity under context
      • has cancelation signal - stops activity on demand
      • has values - key-value list of any data
      • http context is canceled when request is cancelled OR we send back a response
    • channels:
      • sent value is consumed by single receiver
      • cancelation is fanned-out to all receivers
      • just to send some signal, use empty channel chan struct{}
      • sending & receiving from nil channel is forever blocking, BUT nil channel i auto-removed from switch, so it might be useful
      • buffer size
        • keep identical to pool size, keep similar to rate-limiting requirements, remember that > Queues are typically always close to full or close to empty due to the differences in pace between consumers and producers. They very rarely operate in a balanced middle ground where the rate of production and consumption is evenly matched. — and you need to tweak your config
    • any clammed resources and started goroutines must be freed & closed, BUT some close operations aren’t sync, so we also need to do it in blocking manner
    • string formatting can cause deadlocks, when implementing Stringer interface
    • append on slice can cause data race due to shared backing array, thus create copy of it
    • mutex can be used upon critical section only
    • wg.Add can’t be called inside goroutine itself
    • errgroup package can be used to introduce shared cancellable context & error handling upon waitgroups
    • sync can be used only via pointer, otherwise you risk internals duplication
  • std
    • always use time duration for clearness
    • JSON issues
      • embedding structs with Marshaller interface will break default json marshalling
      • time.Time is compared by both monotonic and wall clocks
      • untyped conversions will result in float64 numerics
    • SQL issues
      • .Open won’t always try to establish connections, use Ping
      • configure proper pool size of connections
      • prepared statement are more efficient and secure
      • nullable columns are handled via pointer OR sql.NullXXX types
      • when iterating over .Rows call .Err on each cycle
    • don’t forget to return after responding to http request
    • default HTTP client is missing timeouts & other nifty things
  • testing
  • optimizations