package zerolog import ( "net" "sync" "time" ) var arrayPool = &sync.Pool{ New: func() interface{} { return &Array{ buf: make([]byte, 0, 500), } }, } // Array is used to prepopulate an array of items // which can be re-used to add to log messages. type Array struct { buf []byte } func putArray(a *Array) { // Proper usage of a sync.Pool requires each entry to have approximately // the same memory cost. To obtain this property when the stored type // contains a variably-sized buffer, we add a hard limit on the maximum buffer // to place back in the pool. // // See https://golang.org/issue/23199 const maxSize = 1 << 16 // 64KiB if cap(a.buf) > maxSize { return } arrayPool.Put(a) } // Arr creates an array to be added to an Event or Context. func Arr() *Array { a := arrayPool.Get().(*Array) a.buf = a.buf[:0] return a } // MarshalZerologArray method here is no-op - since data is // already in the needed format. func (*Array) MarshalZerologArray(*Array) { } func (a *Array) write(dst []byte) []byte { dst = enc.AppendArrayStart(dst) if len(a.buf) > 0 { dst = append(dst, a.buf...) } dst = enc.AppendArrayEnd(dst) putArray(a) return dst } // Object marshals an object that implement the LogObjectMarshaler // interface and appends it to the array. func (a *Array) Object(obj LogObjectMarshaler) *Array { e := Dict() obj.MarshalZerologObject(e) e.buf = enc.AppendEndMarker(e.buf) a.buf = append(enc.AppendArrayDelim(a.buf), e.buf...) putEvent(e) return a } // Str appends the val as a string to the array. func (a *Array) Str(val string) *Array { a.buf = enc.AppendString(enc.AppendArrayDelim(a.buf), val) return a } // Bytes appends the val as a string to the array. func (a *Array) Bytes(val []byte) *Array { a.buf = enc.AppendBytes(enc.AppendArrayDelim(a.buf), val) return a } // Hex appends the val as a hex string to the array. func (a *Array) Hex(val []byte) *Array { a.buf = enc.AppendHex(enc.AppendArrayDelim(a.buf), val) return a } // RawJSON adds already encoded JSON to the array. func (a *Array) RawJSON(val []byte) *Array { a.buf = appendJSON(enc.AppendArrayDelim(a.buf), val) return a } // Err serializes and appends the err to the array. func (a *Array) Err(err error) *Array { switch m := ErrorMarshalFunc(err).(type) { case LogObjectMarshaler: e := newEvent(nil, 0) e.buf = e.buf[:0] e.appendObject(m) a.buf = append(enc.AppendArrayDelim(a.buf), e.buf...) putEvent(e) case error: if m == nil || isNilValue(m) { a.buf = enc.AppendNil(enc.AppendArrayDelim(a.buf)) } else { a.buf = enc.AppendString(enc.AppendArrayDelim(a.buf), m.Error()) } case string: a.buf = enc.AppendString(enc.AppendArrayDelim(a.buf), m) default: a.buf = enc.AppendInterface(enc.AppendArrayDelim(a.buf), m) } return a } // Bool appends the val as a bool to the array. func (a *Array) Bool(b bool) *Array { a.buf = enc.AppendBool(enc.AppendArrayDelim(a.buf), b) return a } // Int appends i as a int to the array. func (a *Array) Int(i int) *Array { a.buf = enc.AppendInt(enc.AppendArrayDelim(a.buf), i) return a } // Int8 appends i as a int8 to the array. func (a *Array) Int8(i int8) *Array { a.buf = enc.AppendInt8(enc.AppendArrayDelim(a.buf), i) return a } // Int16 appends i as a int16 to the array. func (a *Array) Int16(i int16) *Array { a.buf = enc.AppendInt16(enc.AppendArrayDelim(a.buf), i) return a } // Int32 appends i as a int32 to the array. func (a *Array) Int32(i int32) *Array { a.buf = enc.AppendInt32(enc.AppendArrayDelim(a.buf), i) return a } // Int64 appends i as a int64 to the array. func (a *Array) Int64(i int64) *Array { a.buf = enc.AppendInt64(enc.AppendArrayDelim(a.buf), i) return a } // Uint appends i as a uint to the array. func (a *Array) Uint(i uint) *Array { a.buf = enc.AppendUint(enc.AppendArrayDelim(a.buf), i) return a } // Uint8 appends i as a uint8 to the array. func (a *Array) Uint8(i uint8) *Array { a.buf = enc.AppendUint8(enc.AppendArrayDelim(a.buf), i) return a } // Uint16 appends i as a uint16 to the array. func (a *Array) Uint16(i uint16) *Array { a.buf = enc.AppendUint16(enc.AppendArrayDelim(a.buf), i) return a } // Uint32 appends i as a uint32 to the array. func (a *Array) Uint32(i uint32) *Array { a.buf = enc.AppendUint32(enc.AppendArrayDelim(a.buf), i) return a } // Uint64 appends i as a uint64 to the array. func (a *Array) Uint64(i uint64) *Array { a.buf = enc.AppendUint64(enc.AppendArrayDelim(a.buf), i) return a } // Float32 appends f as a float32 to the array. func (a *Array) Float32(f float32) *Array { a.buf = enc.AppendFloat32(enc.AppendArrayDelim(a.buf), f, FloatingPointPrecision) return a } // Float64 appends f as a float64 to the array. func (a *Array) Float64(f float64) *Array { a.buf = enc.AppendFloat64(enc.AppendArrayDelim(a.buf), f, FloatingPointPrecision) return a } // Time appends t formatted as string using zerolog.TimeFieldFormat. func (a *Array) Time(t time.Time) *Array { a.buf = enc.AppendTime(enc.AppendArrayDelim(a.buf), t, TimeFieldFormat) return a } // Dur appends d to the array. func (a *Array) Dur(d time.Duration) *Array { a.buf = enc.AppendDuration(enc.AppendArrayDelim(a.buf), d, DurationFieldUnit, DurationFieldInteger, FloatingPointPrecision) return a } // Interface appends i marshaled using reflection. func (a *Array) Interface(i interface{}) *Array { if obj, ok := i.(LogObjectMarshaler); ok { return a.Object(obj) } a.buf = enc.AppendInterface(enc.AppendArrayDelim(a.buf), i) return a } // IPAddr adds IPv4 or IPv6 address to the array func (a *Array) IPAddr(ip net.IP) *Array { a.buf = enc.AppendIPAddr(enc.AppendArrayDelim(a.buf), ip) return a } // IPPrefix adds IPv4 or IPv6 Prefix (IP + mask) to the array func (a *Array) IPPrefix(pfx net.IPNet) *Array { a.buf = enc.AppendIPPrefix(enc.AppendArrayDelim(a.buf), pfx) return a } // MACAddr adds a MAC (Ethernet) address to the array func (a *Array) MACAddr(ha net.HardwareAddr) *Array { a.buf = enc.AppendMACAddr(enc.AppendArrayDelim(a.buf), ha) return a } // Dict adds the dict Event to the array func (a *Array) Dict(dict *Event) *Array { dict.buf = enc.AppendEndMarker(dict.buf) a.buf = append(enc.AppendArrayDelim(a.buf), dict.buf...) return a }
package main import ( "bufio" "errors" "flag" "fmt" "io" "os" "time" "github.com/rs/zerolog" ) func isInputFromPipe() bool { fileInfo, _ := os.Stdin.Stat() return fileInfo.Mode()&os.ModeCharDevice == 0 } func processInput(reader io.Reader, writer io.Writer) error { scanner := bufio.NewScanner(reader) for scanner.Scan() { bytesToWrite := scanner.Bytes() _, err := writer.Write(bytesToWrite) if err != nil { if errors.Is(err, io.EOF) { break } fmt.Printf("%s\n", bytesToWrite) } } return scanner.Err() } func main() { timeFormats := map[string]string{ "default": time.Kitchen, "full": time.RFC1123, } timeFormatFlag := flag.String( "time-format", "default", "Time format, either 'default' or 'full'", ) flag.Parse() timeFormat, ok := timeFormats[*timeFormatFlag] if !ok { panic("Invalid time-format provided") } writer := zerolog.NewConsoleWriter() writer.TimeFormat = timeFormat if isInputFromPipe() { _ = processInput(os.Stdin, writer) } else if flag.NArg() >= 1 { for _, filename := range flag.Args() { // Scan each line from filename and write it into writer reader, err := os.Open(filename) if err != nil { fmt.Printf("%s open: %v", filename, err) os.Exit(1) } if err := processInput(reader, writer); err != nil { fmt.Printf("%s scan: %v", filename, err) os.Exit(1) } } } else { fmt.Println("Usage:") fmt.Println(" app_with_zerolog | 2> >(prettylog)") fmt.Println(" prettylog zerolog_output.jsonl") os.Exit(1) return } }
package zerolog import ( "bytes" "encoding/json" "fmt" "io" "os" "path/filepath" "sort" "strconv" "strings" "sync" "time" "github.com/mattn/go-colorable" ) const ( colorBlack = iota + 30 colorRed colorGreen colorYellow colorBlue colorMagenta colorCyan colorWhite colorBold = 1 colorDarkGray = 90 unknownLevel = "???" ) var ( consoleBufPool = sync.Pool{ New: func() interface{} { return bytes.NewBuffer(make([]byte, 0, 100)) }, } ) const ( consoleDefaultTimeFormat = time.Kitchen ) // Formatter transforms the input into a formatted string. type Formatter func(interface{}) string // ConsoleWriter parses the JSON input and writes it in an // (optionally) colorized, human-friendly format to Out. type ConsoleWriter struct { // Out is the output destination. Out io.Writer // NoColor disables the colorized output. NoColor bool // TimeFormat specifies the format for timestamp in output. TimeFormat string // TimeLocation tells ConsoleWriter’s default FormatTimestamp // how to localize the time. TimeLocation *time.Location // PartsOrder defines the order of parts in output. PartsOrder []string // PartsExclude defines parts to not display in output. PartsExclude []string // FieldsOrder defines the order of contextual fields in output. FieldsOrder []string fieldIsOrdered map[string]int // FieldsExclude defines contextual fields to not display in output. FieldsExclude []string FormatTimestamp Formatter FormatLevel Formatter FormatCaller Formatter FormatMessage Formatter FormatFieldName Formatter FormatFieldValue Formatter FormatErrFieldName Formatter FormatErrFieldValue Formatter FormatExtra func(map[string]interface{}, *bytes.Buffer) error FormatPrepare func(map[string]interface{}) error } // NewConsoleWriter creates and initializes a new ConsoleWriter. func NewConsoleWriter(options ...func(w *ConsoleWriter)) ConsoleWriter { w := ConsoleWriter{ Out: os.Stdout, TimeFormat: consoleDefaultTimeFormat, PartsOrder: consoleDefaultPartsOrder(), } for _, opt := range options { opt(&w) } // Fix color on Windows if w.Out == os.Stdout || w.Out == os.Stderr { w.Out = colorable.NewColorable(w.Out.(*os.File)) } return w } // Write transforms the JSON input with formatters and appends to w.Out. func (w ConsoleWriter) Write(p []byte) (n int, err error) { // Fix color on Windows if w.Out == os.Stdout || w.Out == os.Stderr { w.Out = colorable.NewColorable(w.Out.(*os.File)) } if w.PartsOrder == nil { w.PartsOrder = consoleDefaultPartsOrder() } var buf = consoleBufPool.Get().(*bytes.Buffer) defer func() { buf.Reset() consoleBufPool.Put(buf) }() var evt map[string]interface{} p = decodeIfBinaryToBytes(p) d := json.NewDecoder(bytes.NewReader(p)) d.UseNumber() err = d.Decode(&evt) if err != nil { return n, fmt.Errorf("cannot decode event: %s", err) } if w.FormatPrepare != nil { err = w.FormatPrepare(evt) if err != nil { return n, err } } for _, p := range w.PartsOrder { w.writePart(buf, evt, p) } w.writeFields(evt, buf) if w.FormatExtra != nil { err = w.FormatExtra(evt, buf) if err != nil { return n, err } } err = buf.WriteByte('\n') if err != nil { return n, err } _, err = buf.WriteTo(w.Out) return len(p), err } // Call the underlying writer's Close method if it is an io.Closer. Otherwise // does nothing. func (w ConsoleWriter) Close() error { if closer, ok := w.Out.(io.Closer); ok { return closer.Close() } return nil } // writeFields appends formatted key-value pairs to buf. func (w ConsoleWriter) writeFields(evt map[string]interface{}, buf *bytes.Buffer) { var fields = make([]string, 0, len(evt)) for field := range evt { var isExcluded bool for _, excluded := range w.FieldsExclude { if field == excluded { isExcluded = true break } } if isExcluded { continue } switch field { case LevelFieldName, TimestampFieldName, MessageFieldName, CallerFieldName: continue } fields = append(fields, field) } if len(w.FieldsOrder) > 0 { w.orderFields(fields) } else { sort.Strings(fields) } // Write space only if something has already been written to the buffer, and if there are fields. if buf.Len() > 0 && len(fields) > 0 { buf.WriteByte(' ') } // Move the "error" field to the front ei := sort.Search(len(fields), func(i int) bool { return fields[i] >= ErrorFieldName }) if ei < len(fields) && fields[ei] == ErrorFieldName { fields[ei] = "" fields = append([]string{ErrorFieldName}, fields...) var xfields = make([]string, 0, len(fields)) for _, field := range fields { if field == "" { // Skip empty fields continue } xfields = append(xfields, field) } fields = xfields } for i, field := range fields { var fn Formatter var fv Formatter if field == ErrorFieldName { if w.FormatErrFieldName == nil { fn = consoleDefaultFormatErrFieldName(w.NoColor) } else { fn = w.FormatErrFieldName } if w.FormatErrFieldValue == nil { fv = consoleDefaultFormatErrFieldValue(w.NoColor) } else { fv = w.FormatErrFieldValue } } else { if w.FormatFieldName == nil { fn = consoleDefaultFormatFieldName(w.NoColor) } else { fn = w.FormatFieldName } if w.FormatFieldValue == nil { fv = consoleDefaultFormatFieldValue } else { fv = w.FormatFieldValue } } buf.WriteString(fn(field)) switch fValue := evt[field].(type) { case string: if needsQuote(fValue) { buf.WriteString(fv(strconv.Quote(fValue))) } else { buf.WriteString(fv(fValue)) } case json.Number: buf.WriteString(fv(fValue)) default: b, err := InterfaceMarshalFunc(fValue) if err != nil { fmt.Fprintf(buf, colorize("[error: %v]", colorRed, w.NoColor), err) } else { fmt.Fprint(buf, fv(b)) } } if i < len(fields)-1 { // Skip space for last field buf.WriteByte(' ') } } } // writePart appends a formatted part to buf. func (w ConsoleWriter) writePart(buf *bytes.Buffer, evt map[string]interface{}, p string) { var f Formatter if w.PartsExclude != nil && len(w.PartsExclude) > 0 { for _, exclude := range w.PartsExclude { if exclude == p { return } } } switch p { case LevelFieldName: if w.FormatLevel == nil { f = consoleDefaultFormatLevel(w.NoColor) } else { f = w.FormatLevel } case TimestampFieldName: if w.FormatTimestamp == nil { f = consoleDefaultFormatTimestamp(w.TimeFormat, w.TimeLocation, w.NoColor) } else { f = w.FormatTimestamp } case MessageFieldName: if w.FormatMessage == nil { f = consoleDefaultFormatMessage(w.NoColor, evt[LevelFieldName]) } else { f = w.FormatMessage } case CallerFieldName: if w.FormatCaller == nil { f = consoleDefaultFormatCaller(w.NoColor) } else { f = w.FormatCaller } default: if w.FormatFieldValue == nil { f = consoleDefaultFormatFieldValue } else { f = w.FormatFieldValue } } var s = f(evt[p]) if len(s) > 0 { if buf.Len() > 0 { buf.WriteByte(' ') // Write space only if not the first part } buf.WriteString(s) } } // orderFields takes an array of field names and an array representing field order // and returns an array with any ordered fields at the beginning, in order, // and the remaining fields after in their original order. func (w ConsoleWriter) orderFields(fields []string) { if w.fieldIsOrdered == nil { w.fieldIsOrdered = make(map[string]int) for i, fieldName := range w.FieldsOrder { w.fieldIsOrdered[fieldName] = i } } sort.Slice(fields, func(i, j int) bool { ii, iOrdered := w.fieldIsOrdered[fields[i]] jj, jOrdered := w.fieldIsOrdered[fields[j]] if iOrdered && jOrdered { return ii < jj } if iOrdered { return true } if jOrdered { return false } return fields[i] < fields[j] }) } // needsQuote returns true when the string s should be quoted in output. func needsQuote(s string) bool { for i := range s { if s[i] < 0x20 || s[i] > 0x7e || s[i] == ' ' || s[i] == '\\' || s[i] == '"' { return true } } return false } // colorize returns the string s wrapped in ANSI code c, unless disabled is true or c is 0. func colorize(s interface{}, c int, disabled bool) string { e := os.Getenv("NO_COLOR") if e != "" || c == 0 { disabled = true } if disabled { return fmt.Sprintf("%s", s) } return fmt.Sprintf("\x1b[%dm%v\x1b[0m", c, s) } // ----- DEFAULT FORMATTERS --------------------------------------------------- func consoleDefaultPartsOrder() []string { return []string{ TimestampFieldName, LevelFieldName, CallerFieldName, MessageFieldName, } } func consoleDefaultFormatTimestamp(timeFormat string, location *time.Location, noColor bool) Formatter { if timeFormat == "" { timeFormat = consoleDefaultTimeFormat } if location == nil { location = time.Local } return func(i interface{}) string { t := "<nil>" switch tt := i.(type) { case string: ts, err := time.ParseInLocation(TimeFieldFormat, tt, location) if err != nil { t = tt } else { t = ts.In(location).Format(timeFormat) } case json.Number: i, err := tt.Int64() if err != nil { t = tt.String() } else { var sec, nsec int64 switch TimeFieldFormat { case TimeFormatUnixNano: sec, nsec = 0, i case TimeFormatUnixMicro: sec, nsec = 0, int64(time.Duration(i)*time.Microsecond) case TimeFormatUnixMs: sec, nsec = 0, int64(time.Duration(i)*time.Millisecond) default: sec, nsec = i, 0 } ts := time.Unix(sec, nsec) t = ts.In(location).Format(timeFormat) } } return colorize(t, colorDarkGray, noColor) } } func stripLevel(ll string) string { if len(ll) == 0 { return unknownLevel } if len(ll) > 3 { ll = ll[:3] } return strings.ToUpper(ll) } func consoleDefaultFormatLevel(noColor bool) Formatter { return func(i interface{}) string { if ll, ok := i.(string); ok { level, _ := ParseLevel(ll) fl, ok := FormattedLevels[level] if ok { return colorize(fl, LevelColors[level], noColor) } return stripLevel(ll) } if i == nil { return unknownLevel } return stripLevel(fmt.Sprintf("%s", i)) } } func consoleDefaultFormatCaller(noColor bool) Formatter { return func(i interface{}) string { var c string if cc, ok := i.(string); ok { c = cc } if len(c) > 0 { if cwd, err := os.Getwd(); err == nil { if rel, err := filepath.Rel(cwd, c); err == nil { c = rel } } c = colorize(c, colorBold, noColor) + colorize(" >", colorCyan, noColor) } return c } } func consoleDefaultFormatMessage(noColor bool, level interface{}) Formatter { return func(i interface{}) string { if i == nil || i == "" { return "" } switch level { case LevelInfoValue, LevelWarnValue, LevelErrorValue, LevelFatalValue, LevelPanicValue: return colorize(fmt.Sprintf("%s", i), colorBold, noColor) default: return fmt.Sprintf("%s", i) } } } func consoleDefaultFormatFieldName(noColor bool) Formatter { return func(i interface{}) string { return colorize(fmt.Sprintf("%s=", i), colorCyan, noColor) } } func consoleDefaultFormatFieldValue(i interface{}) string { return fmt.Sprintf("%s", i) } func consoleDefaultFormatErrFieldName(noColor bool) Formatter { return func(i interface{}) string { return colorize(fmt.Sprintf("%s=", i), colorCyan, noColor) } } func consoleDefaultFormatErrFieldValue(noColor bool) Formatter { return func(i interface{}) string { return colorize(colorize(fmt.Sprintf("%s", i), colorBold, noColor), colorRed, noColor) } }
package zerolog import ( "context" "fmt" "io" "math" "net" "time" ) // Context configures a new sub-logger with contextual fields. type Context struct { l Logger } // Logger returns the logger with the context previously set. func (c Context) Logger() Logger { return c.l } // Fields is a helper function to use a map or slice to set fields using type assertion. // Only map[string]interface{} and []interface{} are accepted. []interface{} must // alternate string keys and arbitrary values, and extraneous ones are ignored. func (c Context) Fields(fields interface{}) Context { c.l.context = appendFields(c.l.context, fields, c.l.stack) return c } // Dict adds the field key with the dict to the logger context. func (c Context) Dict(key string, dict *Event) Context { dict.buf = enc.AppendEndMarker(dict.buf) c.l.context = append(enc.AppendKey(c.l.context, key), dict.buf...) putEvent(dict) return c } // Array adds the field key with an array to the event context. // Use zerolog.Arr() to create the array or pass a type that // implement the LogArrayMarshaler interface. func (c Context) Array(key string, arr LogArrayMarshaler) Context { c.l.context = enc.AppendKey(c.l.context, key) if arr, ok := arr.(*Array); ok { c.l.context = arr.write(c.l.context) return c } var a *Array if aa, ok := arr.(*Array); ok { a = aa } else { a = Arr() arr.MarshalZerologArray(a) } c.l.context = a.write(c.l.context) return c } // Object marshals an object that implement the LogObjectMarshaler interface. func (c Context) Object(key string, obj LogObjectMarshaler) Context { e := newEvent(LevelWriterAdapter{io.Discard}, 0) e.Object(key, obj) c.l.context = enc.AppendObjectData(c.l.context, e.buf) putEvent(e) return c } // EmbedObject marshals and Embeds an object that implement the LogObjectMarshaler interface. func (c Context) EmbedObject(obj LogObjectMarshaler) Context { e := newEvent(LevelWriterAdapter{io.Discard}, 0) e.EmbedObject(obj) c.l.context = enc.AppendObjectData(c.l.context, e.buf) putEvent(e) return c } // Str adds the field key with val as a string to the logger context. func (c Context) Str(key, val string) Context { c.l.context = enc.AppendString(enc.AppendKey(c.l.context, key), val) return c } // Strs adds the field key with val as a string to the logger context. func (c Context) Strs(key string, vals []string) Context { c.l.context = enc.AppendStrings(enc.AppendKey(c.l.context, key), vals) return c } // Stringer adds the field key with val.String() (or null if val is nil) to the logger context. func (c Context) Stringer(key string, val fmt.Stringer) Context { if val != nil { c.l.context = enc.AppendString(enc.AppendKey(c.l.context, key), val.String()) return c } c.l.context = enc.AppendInterface(enc.AppendKey(c.l.context, key), nil) return c } // Bytes adds the field key with val as a []byte to the logger context. func (c Context) Bytes(key string, val []byte) Context { c.l.context = enc.AppendBytes(enc.AppendKey(c.l.context, key), val) return c } // Hex adds the field key with val as a hex string to the logger context. func (c Context) Hex(key string, val []byte) Context { c.l.context = enc.AppendHex(enc.AppendKey(c.l.context, key), val) return c } // RawJSON adds already encoded JSON to context. // // No sanity check is performed on b; it must not contain carriage returns and // be valid JSON. func (c Context) RawJSON(key string, b []byte) Context { c.l.context = appendJSON(enc.AppendKey(c.l.context, key), b) return c } // AnErr adds the field key with serialized err to the logger context. func (c Context) AnErr(key string, err error) Context { switch m := ErrorMarshalFunc(err).(type) { case nil: return c case LogObjectMarshaler: return c.Object(key, m) case error: if m == nil || isNilValue(m) { return c } else { return c.Str(key, m.Error()) } case string: return c.Str(key, m) default: return c.Interface(key, m) } } // Errs adds the field key with errs as an array of serialized errors to the // logger context. func (c Context) Errs(key string, errs []error) Context { arr := Arr() for _, err := range errs { switch m := ErrorMarshalFunc(err).(type) { case LogObjectMarshaler: arr = arr.Object(m) case error: if m == nil || isNilValue(m) { arr = arr.Interface(nil) } else { arr = arr.Str(m.Error()) } case string: arr = arr.Str(m) default: arr = arr.Interface(m) } } return c.Array(key, arr) } // Err adds the field "error" with serialized err to the logger context. func (c Context) Err(err error) Context { if c.l.stack && ErrorStackMarshaler != nil { switch m := ErrorStackMarshaler(err).(type) { case nil: case LogObjectMarshaler: c = c.Object(ErrorStackFieldName, m) case error: if m != nil && !isNilValue(m) { c = c.Str(ErrorStackFieldName, m.Error()) } case string: c = c.Str(ErrorStackFieldName, m) default: c = c.Interface(ErrorStackFieldName, m) } } return c.AnErr(ErrorFieldName, err) } // Ctx adds the context.Context to the logger context. The context.Context is // not rendered in the error message, but is made available for hooks to use. // A typical use case is to extract tracing information from the // context.Context. func (c Context) Ctx(ctx context.Context) Context { c.l.ctx = ctx return c } // Bool adds the field key with val as a bool to the logger context. func (c Context) Bool(key string, b bool) Context { c.l.context = enc.AppendBool(enc.AppendKey(c.l.context, key), b) return c } // Bools adds the field key with val as a []bool to the logger context. func (c Context) Bools(key string, b []bool) Context { c.l.context = enc.AppendBools(enc.AppendKey(c.l.context, key), b) return c } // Int adds the field key with i as a int to the logger context. func (c Context) Int(key string, i int) Context { c.l.context = enc.AppendInt(enc.AppendKey(c.l.context, key), i) return c } // Ints adds the field key with i as a []int to the logger context. func (c Context) Ints(key string, i []int) Context { c.l.context = enc.AppendInts(enc.AppendKey(c.l.context, key), i) return c } // Int8 adds the field key with i as a int8 to the logger context. func (c Context) Int8(key string, i int8) Context { c.l.context = enc.AppendInt8(enc.AppendKey(c.l.context, key), i) return c } // Ints8 adds the field key with i as a []int8 to the logger context. func (c Context) Ints8(key string, i []int8) Context { c.l.context = enc.AppendInts8(enc.AppendKey(c.l.context, key), i) return c } // Int16 adds the field key with i as a int16 to the logger context. func (c Context) Int16(key string, i int16) Context { c.l.context = enc.AppendInt16(enc.AppendKey(c.l.context, key), i) return c } // Ints16 adds the field key with i as a []int16 to the logger context. func (c Context) Ints16(key string, i []int16) Context { c.l.context = enc.AppendInts16(enc.AppendKey(c.l.context, key), i) return c } // Int32 adds the field key with i as a int32 to the logger context. func (c Context) Int32(key string, i int32) Context { c.l.context = enc.AppendInt32(enc.AppendKey(c.l.context, key), i) return c } // Ints32 adds the field key with i as a []int32 to the logger context. func (c Context) Ints32(key string, i []int32) Context { c.l.context = enc.AppendInts32(enc.AppendKey(c.l.context, key), i) return c } // Int64 adds the field key with i as a int64 to the logger context. func (c Context) Int64(key string, i int64) Context { c.l.context = enc.AppendInt64(enc.AppendKey(c.l.context, key), i) return c } // Ints64 adds the field key with i as a []int64 to the logger context. func (c Context) Ints64(key string, i []int64) Context { c.l.context = enc.AppendInts64(enc.AppendKey(c.l.context, key), i) return c } // Uint adds the field key with i as a uint to the logger context. func (c Context) Uint(key string, i uint) Context { c.l.context = enc.AppendUint(enc.AppendKey(c.l.context, key), i) return c } // Uints adds the field key with i as a []uint to the logger context. func (c Context) Uints(key string, i []uint) Context { c.l.context = enc.AppendUints(enc.AppendKey(c.l.context, key), i) return c } // Uint8 adds the field key with i as a uint8 to the logger context. func (c Context) Uint8(key string, i uint8) Context { c.l.context = enc.AppendUint8(enc.AppendKey(c.l.context, key), i) return c } // Uints8 adds the field key with i as a []uint8 to the logger context. func (c Context) Uints8(key string, i []uint8) Context { c.l.context = enc.AppendUints8(enc.AppendKey(c.l.context, key), i) return c } // Uint16 adds the field key with i as a uint16 to the logger context. func (c Context) Uint16(key string, i uint16) Context { c.l.context = enc.AppendUint16(enc.AppendKey(c.l.context, key), i) return c } // Uints16 adds the field key with i as a []uint16 to the logger context. func (c Context) Uints16(key string, i []uint16) Context { c.l.context = enc.AppendUints16(enc.AppendKey(c.l.context, key), i) return c } // Uint32 adds the field key with i as a uint32 to the logger context. func (c Context) Uint32(key string, i uint32) Context { c.l.context = enc.AppendUint32(enc.AppendKey(c.l.context, key), i) return c } // Uints32 adds the field key with i as a []uint32 to the logger context. func (c Context) Uints32(key string, i []uint32) Context { c.l.context = enc.AppendUints32(enc.AppendKey(c.l.context, key), i) return c } // Uint64 adds the field key with i as a uint64 to the logger context. func (c Context) Uint64(key string, i uint64) Context { c.l.context = enc.AppendUint64(enc.AppendKey(c.l.context, key), i) return c } // Uints64 adds the field key with i as a []uint64 to the logger context. func (c Context) Uints64(key string, i []uint64) Context { c.l.context = enc.AppendUints64(enc.AppendKey(c.l.context, key), i) return c } // Float32 adds the field key with f as a float32 to the logger context. func (c Context) Float32(key string, f float32) Context { c.l.context = enc.AppendFloat32(enc.AppendKey(c.l.context, key), f, FloatingPointPrecision) return c } // Floats32 adds the field key with f as a []float32 to the logger context. func (c Context) Floats32(key string, f []float32) Context { c.l.context = enc.AppendFloats32(enc.AppendKey(c.l.context, key), f, FloatingPointPrecision) return c } // Float64 adds the field key with f as a float64 to the logger context. func (c Context) Float64(key string, f float64) Context { c.l.context = enc.AppendFloat64(enc.AppendKey(c.l.context, key), f, FloatingPointPrecision) return c } // Floats64 adds the field key with f as a []float64 to the logger context. func (c Context) Floats64(key string, f []float64) Context { c.l.context = enc.AppendFloats64(enc.AppendKey(c.l.context, key), f, FloatingPointPrecision) return c } type timestampHook struct{} func (ts timestampHook) Run(e *Event, level Level, msg string) { e.Timestamp() } var th = timestampHook{} // Timestamp adds the current local time to the logger context with the "time" key, formatted using zerolog.TimeFieldFormat. // To customize the key name, change zerolog.TimestampFieldName. // To customize the time format, change zerolog.TimeFieldFormat. // // NOTE: It won't dedupe the "time" key if the *Context has one already. func (c Context) Timestamp() Context { c.l = c.l.Hook(th) return c } // Time adds the field key with t formatted as string using zerolog.TimeFieldFormat. func (c Context) Time(key string, t time.Time) Context { c.l.context = enc.AppendTime(enc.AppendKey(c.l.context, key), t, TimeFieldFormat) return c } // Times adds the field key with t formatted as string using zerolog.TimeFieldFormat. func (c Context) Times(key string, t []time.Time) Context { c.l.context = enc.AppendTimes(enc.AppendKey(c.l.context, key), t, TimeFieldFormat) return c } // Dur adds the fields key with d divided by unit and stored as a float. func (c Context) Dur(key string, d time.Duration) Context { c.l.context = enc.AppendDuration(enc.AppendKey(c.l.context, key), d, DurationFieldUnit, DurationFieldInteger, FloatingPointPrecision) return c } // Durs adds the fields key with d divided by unit and stored as a float. func (c Context) Durs(key string, d []time.Duration) Context { c.l.context = enc.AppendDurations(enc.AppendKey(c.l.context, key), d, DurationFieldUnit, DurationFieldInteger, FloatingPointPrecision) return c } // Interface adds the field key with obj marshaled using reflection. func (c Context) Interface(key string, i interface{}) Context { if obj, ok := i.(LogObjectMarshaler); ok { return c.Object(key, obj) } c.l.context = enc.AppendInterface(enc.AppendKey(c.l.context, key), i) return c } // Type adds the field key with val's type using reflection. func (c Context) Type(key string, val interface{}) Context { c.l.context = enc.AppendType(enc.AppendKey(c.l.context, key), val) return c } // Any is a wrapper around Context.Interface. func (c Context) Any(key string, i interface{}) Context { return c.Interface(key, i) } // Reset removes all the context fields. func (c Context) Reset() Context { c.l.context = enc.AppendBeginMarker(make([]byte, 0, 500)) return c } type callerHook struct { callerSkipFrameCount int } func newCallerHook(skipFrameCount int) callerHook { return callerHook{callerSkipFrameCount: skipFrameCount} } func (ch callerHook) Run(e *Event, level Level, msg string) { switch ch.callerSkipFrameCount { case useGlobalSkipFrameCount: // Extra frames to skip (added by hook infra). e.caller(CallerSkipFrameCount + contextCallerSkipFrameCount) default: // Extra frames to skip (added by hook infra). e.caller(ch.callerSkipFrameCount + contextCallerSkipFrameCount) } } // useGlobalSkipFrameCount acts as a flag to informat callerHook.Run // to use the global CallerSkipFrameCount. const useGlobalSkipFrameCount = math.MinInt32 // ch is the default caller hook using the global CallerSkipFrameCount. var ch = newCallerHook(useGlobalSkipFrameCount) // Caller adds the file:line of the caller with the zerolog.CallerFieldName key. func (c Context) Caller() Context { c.l = c.l.Hook(ch) return c } // CallerWithSkipFrameCount adds the file:line of the caller with the zerolog.CallerFieldName key. // The specified skipFrameCount int will override the global CallerSkipFrameCount for this context's respective logger. // If set to -1 the global CallerSkipFrameCount will be used. func (c Context) CallerWithSkipFrameCount(skipFrameCount int) Context { c.l = c.l.Hook(newCallerHook(skipFrameCount)) return c } // Stack enables stack trace printing for the error passed to Err(). func (c Context) Stack() Context { c.l.stack = true return c } // IPAddr adds IPv4 or IPv6 Address to the context func (c Context) IPAddr(key string, ip net.IP) Context { c.l.context = enc.AppendIPAddr(enc.AppendKey(c.l.context, key), ip) return c } // IPPrefix adds IPv4 or IPv6 Prefix (address and mask) to the context func (c Context) IPPrefix(key string, pfx net.IPNet) Context { c.l.context = enc.AppendIPPrefix(enc.AppendKey(c.l.context, key), pfx) return c } // MACAddr adds MAC address to the context func (c Context) MACAddr(key string, ha net.HardwareAddr) Context { c.l.context = enc.AppendMACAddr(enc.AppendKey(c.l.context, key), ha) return c }
package zerolog import ( "context" ) var disabledLogger *Logger func init() { SetGlobalLevel(TraceLevel) l := Nop() disabledLogger = &l } type ctxKey struct{} // WithContext returns a copy of ctx with the receiver attached. The Logger // attached to the provided Context (if any) will not be effected. If the // receiver's log level is Disabled it will only be attached to the returned // Context if the provided Context has a previously attached Logger. If the // provided Context has no attached Logger, a Disabled Logger will not be // attached. // // Note: to modify the existing Logger attached to a Context (instead of // replacing it in a new Context), use UpdateContext with the following // notation: // // ctx := r.Context() // l := zerolog.Ctx(ctx) // l.UpdateContext(func(c Context) Context { // return c.Str("bar", "baz") // }) // func (l Logger) WithContext(ctx context.Context) context.Context { if _, ok := ctx.Value(ctxKey{}).(*Logger); !ok && l.level == Disabled { // Do not store disabled logger. return ctx } return context.WithValue(ctx, ctxKey{}, &l) } // Ctx returns the Logger associated with the ctx. If no logger // is associated, DefaultContextLogger is returned, unless DefaultContextLogger // is nil, in which case a disabled logger is returned. func Ctx(ctx context.Context) *Logger { if l, ok := ctx.Value(ctxKey{}).(*Logger); ok { return l } else if l = DefaultContextLogger; l != nil { return l } return disabledLogger }
// Package diode provides a thread-safe, lock-free, non-blocking io.Writer // wrapper. package diode import ( "context" "io" "sync" "time" "github.com/rs/zerolog/diode/internal/diodes" ) var bufPool = &sync.Pool{ New: func() interface{} { return make([]byte, 0, 500) }, } type Alerter func(missed int) type diodeFetcher interface { diodes.Diode Next() diodes.GenericDataType } // Writer is a io.Writer wrapper that uses a diode to make Write lock-free, // non-blocking and thread safe. type Writer struct { w io.Writer d diodeFetcher c context.CancelFunc done chan struct{} } // NewWriter creates a writer wrapping w with a many-to-one diode in order to // never block log producers and drop events if the writer can't keep up with // the flow of data. // // Use a diode.Writer when // // wr := diode.NewWriter(w, 1000, 0, func(missed int) { // log.Printf("Dropped %d messages", missed) // }) // log := zerolog.New(wr) // // If pollInterval is greater than 0, a poller is used otherwise a waiter is // used. // // See code.cloudfoundry.org/go-diodes for more info on diode. func NewWriter(w io.Writer, size int, pollInterval time.Duration, f Alerter) Writer { ctx, cancel := context.WithCancel(context.Background()) dw := Writer{ w: w, c: cancel, done: make(chan struct{}), } if f == nil { f = func(int) {} } d := diodes.NewManyToOne(size, diodes.AlertFunc(f)) if pollInterval > 0 { dw.d = diodes.NewPoller(d, diodes.WithPollingInterval(pollInterval), diodes.WithPollingContext(ctx)) } else { dw.d = diodes.NewWaiter(d, diodes.WithWaiterContext(ctx)) } go dw.poll() return dw } func (dw Writer) Write(p []byte) (n int, err error) { // p is pooled in zerolog so we can't hold it passed this call, hence the // copy. p = append(bufPool.Get().([]byte), p...) dw.d.Set(diodes.GenericDataType(&p)) return len(p), nil } // Close releases the diode poller and call Close on the wrapped writer if // io.Closer is implemented. func (dw Writer) Close() error { dw.c() <-dw.done if w, ok := dw.w.(io.Closer); ok { return w.Close() } return nil } func (dw Writer) poll() { defer close(dw.done) for { d := dw.d.Next() if d == nil { return } p := *(*[]byte)(d) dw.w.Write(p) // Proper usage of a sync.Pool requires each entry to have approximately // the same memory cost. To obtain this property when the stored type // contains a variably-sized buffer, we add a hard limit on the maximum buffer // to place back in the pool. // // See https://golang.org/issue/23199 const maxSize = 1 << 16 // 64KiB if cap(p) <= maxSize { bufPool.Put(p[:0]) } } }
package diodes import ( "log" "sync/atomic" "unsafe" ) // ManyToOne diode is optimal for many writers (go-routines B-n) and a single // reader (go-routine A). It is not thread safe for multiple readers. type ManyToOne struct { writeIndex uint64 readIndex uint64 buffer []unsafe.Pointer alerter Alerter } // NewManyToOne creates a new diode (ring buffer). The ManyToOne diode // is optimized for many writers (on go-routines B-n) and a single reader // (on go-routine A). The alerter is invoked on the read's go-routine. It is // called when it notices that the writer go-routine has passed it and wrote // over data. A nil can be used to ignore alerts. func NewManyToOne(size int, alerter Alerter) *ManyToOne { if alerter == nil { alerter = AlertFunc(func(int) {}) } d := &ManyToOne{ buffer: make([]unsafe.Pointer, size), alerter: alerter, } // Start write index at the value before 0 // to allow the first write to use AddUint64 // and still have a beginning index of 0 d.writeIndex = ^d.writeIndex return d } // Set sets the data in the next slot of the ring buffer. func (d *ManyToOne) Set(data GenericDataType) { for { writeIndex := atomic.AddUint64(&d.writeIndex, 1) idx := writeIndex % uint64(len(d.buffer)) old := atomic.LoadPointer(&d.buffer[idx]) if old != nil && (*bucket)(old) != nil && (*bucket)(old).seq > writeIndex-uint64(len(d.buffer)) { log.Println("Diode set collision: consider using a larger diode") continue } newBucket := &bucket{ data: data, seq: writeIndex, } if !atomic.CompareAndSwapPointer(&d.buffer[idx], old, unsafe.Pointer(newBucket)) { log.Println("Diode set collision: consider using a larger diode") continue } return } } // TryNext will attempt to read from the next slot of the ring buffer. // If there is no data available, it will return (nil, false). func (d *ManyToOne) TryNext() (data GenericDataType, ok bool) { // Read a value from the ring buffer based on the readIndex. idx := d.readIndex % uint64(len(d.buffer)) result := (*bucket)(atomic.SwapPointer(&d.buffer[idx], nil)) // When the result is nil that means the writer has not had the // opportunity to write a value into the diode. This value must be ignored // and the read head must not increment. if result == nil { return nil, false } // When the seq value is less than the current read index that means a // value was read from idx that was previously written but since has // been dropped. This value must be ignored and the read head must not // increment. // // The simulation for this scenario assumes the fast forward occurred as // detailed below. // // 5. The reader reads again getting seq 5. It then reads again expecting // seq 6 but gets seq 2. This is a read of a stale value that was // effectively "dropped" so the read fails and the read head stays put. // `| 4 | 5 | 2 | 3 |` r: 7, w: 6 // if result.seq < d.readIndex { return nil, false } // When the seq value is greater than the current read index that means a // value was read from idx that overwrote the value that was expected to // be at this idx. This happens when the writer has lapped the reader. The // reader needs to catch up to the writer so it moves its write head to // the new seq, effectively dropping the messages that were not read in // between the two values. // // Here is a simulation of this scenario: // // 1. Both the read and write heads start at 0. // `| nil | nil | nil | nil |` r: 0, w: 0 // 2. The writer fills the buffer. // `| 0 | 1 | 2 | 3 |` r: 0, w: 4 // 3. The writer laps the read head. // `| 4 | 5 | 2 | 3 |` r: 0, w: 6 // 4. The reader reads the first value, expecting a seq of 0 but reads 4, // this forces the reader to fast forward to 5. // `| 4 | 5 | 2 | 3 |` r: 5, w: 6 // if result.seq > d.readIndex { dropped := result.seq - d.readIndex d.readIndex = result.seq d.alerter.Alert(int(dropped)) } // Only increment read index if a regular read occurred (where seq was // equal to readIndex) or a value was read that caused a fast forward // (where seq was greater than readIndex). // d.readIndex++ return result.data, true }
package diodes import ( "sync/atomic" "unsafe" ) // GenericDataType is the data type the diodes operate on. type GenericDataType unsafe.Pointer // Alerter is used to report how many values were overwritten since the // last write. type Alerter interface { Alert(missed int) } // AlertFunc type is an adapter to allow the use of ordinary functions as // Alert handlers. type AlertFunc func(missed int) // Alert calls f(missed) func (f AlertFunc) Alert(missed int) { f(missed) } type bucket struct { data GenericDataType seq uint64 // seq is the recorded write index at the time of writing } // OneToOne diode is meant to be used by a single reader and a single writer. // It is not thread safe if used otherwise. type OneToOne struct { writeIndex uint64 readIndex uint64 buffer []unsafe.Pointer alerter Alerter } // NewOneToOne creates a new diode is meant to be used by a single reader and // a single writer. The alerter is invoked on the read's go-routine. It is // called when it notices that the writer go-routine has passed it and wrote // over data. A nil can be used to ignore alerts. func NewOneToOne(size int, alerter Alerter) *OneToOne { if alerter == nil { alerter = AlertFunc(func(int) {}) } return &OneToOne{ buffer: make([]unsafe.Pointer, size), alerter: alerter, } } // Set sets the data in the next slot of the ring buffer. func (d *OneToOne) Set(data GenericDataType) { idx := d.writeIndex % uint64(len(d.buffer)) newBucket := &bucket{ data: data, seq: d.writeIndex, } d.writeIndex++ atomic.StorePointer(&d.buffer[idx], unsafe.Pointer(newBucket)) } // TryNext will attempt to read from the next slot of the ring buffer. // If there is no data available, it will return (nil, false). func (d *OneToOne) TryNext() (data GenericDataType, ok bool) { // Read a value from the ring buffer based on the readIndex. idx := d.readIndex % uint64(len(d.buffer)) result := (*bucket)(atomic.SwapPointer(&d.buffer[idx], nil)) // When the result is nil that means the writer has not had the // opportunity to write a value into the diode. This value must be ignored // and the read head must not increment. if result == nil { return nil, false } // When the seq value is less than the current read index that means a // value was read from idx that was previously written but since has // been dropped. This value must be ignored and the read head must not // increment. // // The simulation for this scenario assumes the fast forward occurred as // detailed below. // // 5. The reader reads again getting seq 5. It then reads again expecting // seq 6 but gets seq 2. This is a read of a stale value that was // effectively "dropped" so the read fails and the read head stays put. // `| 4 | 5 | 2 | 3 |` r: 7, w: 6 // if result.seq < d.readIndex { return nil, false } // When the seq value is greater than the current read index that means a // value was read from idx that overwrote the value that was expected to // be at this idx. This happens when the writer has lapped the reader. The // reader needs to catch up to the writer so it moves its write head to // the new seq, effectively dropping the messages that were not read in // between the two values. // // Here is a simulation of this scenario: // // 1. Both the read and write heads start at 0. // `| nil | nil | nil | nil |` r: 0, w: 0 // 2. The writer fills the buffer. // `| 0 | 1 | 2 | 3 |` r: 0, w: 4 // 3. The writer laps the read head. // `| 4 | 5 | 2 | 3 |` r: 0, w: 6 // 4. The reader reads the first value, expecting a seq of 0 but reads 4, // this forces the reader to fast forward to 5. // `| 4 | 5 | 2 | 3 |` r: 5, w: 6 // if result.seq > d.readIndex { dropped := result.seq - d.readIndex d.readIndex = result.seq d.alerter.Alert(int(dropped)) } // Only increment read index if a regular read occurred (where seq was // equal to readIndex) or a value was read that caused a fast forward // (where seq was greater than readIndex). d.readIndex++ return result.data, true }
package diodes import ( "context" "time" ) // Diode is any implementation of a diode. type Diode interface { Set(GenericDataType) TryNext() (GenericDataType, bool) } // Poller will poll a diode until a value is available. type Poller struct { Diode interval time.Duration ctx context.Context } // PollerConfigOption can be used to setup the poller. type PollerConfigOption func(*Poller) // WithPollingInterval sets the interval at which the diode is queried // for new data. The default is 10ms. func WithPollingInterval(interval time.Duration) PollerConfigOption { return func(c *Poller) { c.interval = interval } } // WithPollingContext sets the context to cancel any retrieval (Next()). It // will not change any results for adding data (Set()). Default is // context.Background(). func WithPollingContext(ctx context.Context) PollerConfigOption { return func(c *Poller) { c.ctx = ctx } } // NewPoller returns a new Poller that wraps the given diode. func NewPoller(d Diode, opts ...PollerConfigOption) *Poller { p := &Poller{ Diode: d, interval: 10 * time.Millisecond, ctx: context.Background(), } for _, o := range opts { o(p) } return p } // Next polls the diode until data is available or until the context is done. // If the context is done, then nil will be returned. func (p *Poller) Next() GenericDataType { for { data, ok := p.Diode.TryNext() if !ok { if p.isDone() { return nil } time.Sleep(p.interval) continue } return data } } func (p *Poller) isDone() bool { select { case <-p.ctx.Done(): return true default: return false } }
package diodes import ( "context" "sync" ) // Waiter will use a conditional mutex to alert the reader to when data is // available. type Waiter struct { Diode mu sync.Mutex c *sync.Cond ctx context.Context } // WaiterConfigOption can be used to setup the waiter. type WaiterConfigOption func(*Waiter) // WithWaiterContext sets the context to cancel any retrieval (Next()). It // will not change any results for adding data (Set()). Default is // context.Background(). func WithWaiterContext(ctx context.Context) WaiterConfigOption { return func(c *Waiter) { c.ctx = ctx } } // NewWaiter returns a new Waiter that wraps the given diode. func NewWaiter(d Diode, opts ...WaiterConfigOption) *Waiter { w := new(Waiter) w.Diode = d w.c = sync.NewCond(&w.mu) w.ctx = context.Background() for _, opt := range opts { opt(w) } go func() { <-w.ctx.Done() // Mutex is strictly necessary here to avoid a race in Next() (between // w.isDone() and w.c.Wait()) and w.c.Broadcast() here. w.mu.Lock() w.c.Broadcast() w.mu.Unlock() }() return w } // Set invokes the wrapped diode's Set with the given data and uses Broadcast // to wake up any readers. func (w *Waiter) Set(data GenericDataType) { w.Diode.Set(data) w.c.Broadcast() } // Next returns the next data point on the wrapped diode. If there is not any // new data, it will Wait for set to be called or the context to be done. // If the context is done, then nil will be returned. func (w *Waiter) Next() GenericDataType { w.mu.Lock() defer w.mu.Unlock() for { data, ok := w.Diode.TryNext() if !ok { if w.isDone() { return nil } w.c.Wait() continue } return data } } func (w *Waiter) isDone() bool { select { case <-w.ctx.Done(): return true default: return false } }
// +build !binary_log package zerolog // encoder_json.go file contains bindings to generate // JSON encoded byte stream. import ( "encoding/base64" "github.com/rs/zerolog/internal/json" ) var ( _ encoder = (*json.Encoder)(nil) enc = json.Encoder{} ) func init() { // using closure to reflect the changes at runtime. json.JSONMarshalFunc = func(v interface{}) ([]byte, error) { return InterfaceMarshalFunc(v) } } func appendJSON(dst []byte, j []byte) []byte { return append(dst, j...) } func appendCBOR(dst []byte, cbor []byte) []byte { dst = append(dst, []byte("\"data:application/cbor;base64,")...) l := len(dst) enc := base64.StdEncoding n := enc.EncodedLen(len(cbor)) for i := 0; i < n; i++ { dst = append(dst, '.') } enc.Encode(dst[l:], cbor) return append(dst, '"') } func decodeIfBinaryToString(in []byte) string { return string(in) } func decodeObjectToStr(in []byte) string { return string(in) } func decodeIfBinaryToBytes(in []byte) []byte { return in }
package zerolog import ( "context" "fmt" "net" "os" "runtime" "sync" "time" ) var eventPool = &sync.Pool{ New: func() interface{} { return &Event{ buf: make([]byte, 0, 500), } }, } // Event represents a log event. It is instanced by one of the level method of // Logger and finalized by the Msg or Msgf method. type Event struct { buf []byte w LevelWriter level Level done func(msg string) stack bool // enable error stack trace ch []Hook // hooks from context skipFrame int // The number of additional frames to skip when printing the caller. ctx context.Context // Optional Go context for event } func putEvent(e *Event) { // Proper usage of a sync.Pool requires each entry to have approximately // the same memory cost. To obtain this property when the stored type // contains a variably-sized buffer, we add a hard limit on the maximum buffer // to place back in the pool. // // See https://golang.org/issue/23199 const maxSize = 1 << 16 // 64KiB if cap(e.buf) > maxSize { return } eventPool.Put(e) } // LogObjectMarshaler provides a strongly-typed and encoding-agnostic interface // to be implemented by types used with Event/Context's Object methods. type LogObjectMarshaler interface { MarshalZerologObject(e *Event) } // LogArrayMarshaler provides a strongly-typed and encoding-agnostic interface // to be implemented by types used with Event/Context's Array methods. type LogArrayMarshaler interface { MarshalZerologArray(a *Array) } func newEvent(w LevelWriter, level Level) *Event { e := eventPool.Get().(*Event) e.buf = e.buf[:0] e.ch = nil e.buf = enc.AppendBeginMarker(e.buf) e.w = w e.level = level e.stack = false e.skipFrame = 0 return e } func (e *Event) write() (err error) { if e == nil { return nil } if e.level != Disabled { e.buf = enc.AppendEndMarker(e.buf) e.buf = enc.AppendLineBreak(e.buf) if e.w != nil { _, err = e.w.WriteLevel(e.level, e.buf) } } putEvent(e) return } // Enabled return false if the *Event is going to be filtered out by // log level or sampling. func (e *Event) Enabled() bool { return e != nil && e.level != Disabled } // Discard disables the event so Msg(f) won't print it. func (e *Event) Discard() *Event { if e == nil { return e } e.level = Disabled return nil } // Msg sends the *Event with msg added as the message field if not empty. // // NOTICE: once this method is called, the *Event should be disposed. // Calling Msg twice can have unexpected result. func (e *Event) Msg(msg string) { if e == nil { return } e.msg(msg) } // Send is equivalent to calling Msg(""). // // NOTICE: once this method is called, the *Event should be disposed. func (e *Event) Send() { if e == nil { return } e.msg("") } // Msgf sends the event with formatted msg added as the message field if not empty. // // NOTICE: once this method is called, the *Event should be disposed. // Calling Msgf twice can have unexpected result. func (e *Event) Msgf(format string, v ...interface{}) { if e == nil { return } e.msg(fmt.Sprintf(format, v...)) } func (e *Event) MsgFunc(createMsg func() string) { if e == nil { return } e.msg(createMsg()) } func (e *Event) msg(msg string) { for _, hook := range e.ch { hook.Run(e, e.level, msg) } if msg != "" { e.buf = enc.AppendString(enc.AppendKey(e.buf, MessageFieldName), msg) } if e.done != nil { defer e.done(msg) } if err := e.write(); err != nil { if ErrorHandler != nil { ErrorHandler(err) } else { fmt.Fprintf(os.Stderr, "zerolog: could not write event: %v\n", err) } } } // Fields is a helper function to use a map or slice to set fields using type assertion. // Only map[string]interface{} and []interface{} are accepted. []interface{} must // alternate string keys and arbitrary values, and extraneous ones are ignored. func (e *Event) Fields(fields interface{}) *Event { if e == nil { return e } e.buf = appendFields(e.buf, fields, e.stack) return e } // Dict adds the field key with a dict to the event context. // Use zerolog.Dict() to create the dictionary. func (e *Event) Dict(key string, dict *Event) *Event { if e == nil { return e } dict.buf = enc.AppendEndMarker(dict.buf) e.buf = append(enc.AppendKey(e.buf, key), dict.buf...) putEvent(dict) return e } // Dict creates an Event to be used with the *Event.Dict method. // Call usual field methods like Str, Int etc to add fields to this // event and give it as argument the *Event.Dict method. func Dict() *Event { return newEvent(nil, 0) } // Array adds the field key with an array to the event context. // Use zerolog.Arr() to create the array or pass a type that // implement the LogArrayMarshaler interface. func (e *Event) Array(key string, arr LogArrayMarshaler) *Event { if e == nil { return e } e.buf = enc.AppendKey(e.buf, key) var a *Array if aa, ok := arr.(*Array); ok { a = aa } else { a = Arr() arr.MarshalZerologArray(a) } e.buf = a.write(e.buf) return e } func (e *Event) appendObject(obj LogObjectMarshaler) { e.buf = enc.AppendBeginMarker(e.buf) obj.MarshalZerologObject(e) e.buf = enc.AppendEndMarker(e.buf) } // Object marshals an object that implement the LogObjectMarshaler interface. func (e *Event) Object(key string, obj LogObjectMarshaler) *Event { if e == nil { return e } e.buf = enc.AppendKey(e.buf, key) if obj == nil { e.buf = enc.AppendNil(e.buf) return e } e.appendObject(obj) return e } // Func allows an anonymous func to run only if the event is enabled. func (e *Event) Func(f func(e *Event)) *Event { if e != nil && e.Enabled() { f(e) } return e } // EmbedObject marshals an object that implement the LogObjectMarshaler interface. func (e *Event) EmbedObject(obj LogObjectMarshaler) *Event { if e == nil { return e } if obj == nil { return e } obj.MarshalZerologObject(e) return e } // Str adds the field key with val as a string to the *Event context. func (e *Event) Str(key, val string) *Event { if e == nil { return e } e.buf = enc.AppendString(enc.AppendKey(e.buf, key), val) return e } // Strs adds the field key with vals as a []string to the *Event context. func (e *Event) Strs(key string, vals []string) *Event { if e == nil { return e } e.buf = enc.AppendStrings(enc.AppendKey(e.buf, key), vals) return e } // Stringer adds the field key with val.String() (or null if val is nil) // to the *Event context. func (e *Event) Stringer(key string, val fmt.Stringer) *Event { if e == nil { return e } e.buf = enc.AppendStringer(enc.AppendKey(e.buf, key), val) return e } // Stringers adds the field key with vals where each individual val // is used as val.String() (or null if val is empty) to the *Event // context. func (e *Event) Stringers(key string, vals []fmt.Stringer) *Event { if e == nil { return e } e.buf = enc.AppendStringers(enc.AppendKey(e.buf, key), vals) return e } // Bytes adds the field key with val as a string to the *Event context. // // Runes outside of normal ASCII ranges will be hex-encoded in the resulting // JSON. func (e *Event) Bytes(key string, val []byte) *Event { if e == nil { return e } e.buf = enc.AppendBytes(enc.AppendKey(e.buf, key), val) return e } // Hex adds the field key with val as a hex string to the *Event context. func (e *Event) Hex(key string, val []byte) *Event { if e == nil { return e } e.buf = enc.AppendHex(enc.AppendKey(e.buf, key), val) return e } // RawJSON adds already encoded JSON to the log line under key. // // No sanity check is performed on b; it must not contain carriage returns and // be valid JSON. func (e *Event) RawJSON(key string, b []byte) *Event { if e == nil { return e } e.buf = appendJSON(enc.AppendKey(e.buf, key), b) return e } // RawCBOR adds already encoded CBOR to the log line under key. // // No sanity check is performed on b // Note: The full featureset of CBOR is supported as data will not be mapped to json but stored as data-url func (e *Event) RawCBOR(key string, b []byte) *Event { if e == nil { return e } e.buf = appendCBOR(enc.AppendKey(e.buf, key), b) return e } // AnErr adds the field key with serialized err to the *Event context. // If err is nil, no field is added. func (e *Event) AnErr(key string, err error) *Event { if e == nil { return e } switch m := ErrorMarshalFunc(err).(type) { case nil: return e case LogObjectMarshaler: return e.Object(key, m) case error: if m == nil || isNilValue(m) { return e } else { return e.Str(key, m.Error()) } case string: return e.Str(key, m) default: return e.Interface(key, m) } } // Errs adds the field key with errs as an array of serialized errors to the // *Event context. func (e *Event) Errs(key string, errs []error) *Event { if e == nil { return e } arr := Arr() for _, err := range errs { switch m := ErrorMarshalFunc(err).(type) { case LogObjectMarshaler: arr = arr.Object(m) case error: arr = arr.Err(m) case string: arr = arr.Str(m) default: arr = arr.Interface(m) } } return e.Array(key, arr) } // Err adds the field "error" with serialized err to the *Event context. // If err is nil, no field is added. // // To customize the key name, change zerolog.ErrorFieldName. // // If Stack() has been called before and zerolog.ErrorStackMarshaler is defined, // the err is passed to ErrorStackMarshaler and the result is appended to the // zerolog.ErrorStackFieldName. func (e *Event) Err(err error) *Event { if e == nil { return e } if e.stack && ErrorStackMarshaler != nil { switch m := ErrorStackMarshaler(err).(type) { case nil: case LogObjectMarshaler: e.Object(ErrorStackFieldName, m) case error: if m != nil && !isNilValue(m) { e.Str(ErrorStackFieldName, m.Error()) } case string: e.Str(ErrorStackFieldName, m) default: e.Interface(ErrorStackFieldName, m) } } return e.AnErr(ErrorFieldName, err) } // Stack enables stack trace printing for the error passed to Err(). // // ErrorStackMarshaler must be set for this method to do something. func (e *Event) Stack() *Event { if e != nil { e.stack = true } return e } // Ctx adds the Go Context to the *Event context. The context is not rendered // in the output message, but is available to hooks and to Func() calls via the // GetCtx() accessor. A typical use case is to extract tracing information from // the Go Ctx. func (e *Event) Ctx(ctx context.Context) *Event { if e != nil { e.ctx = ctx } return e } // GetCtx retrieves the Go context.Context which is optionally stored in the // Event. This allows Hooks and functions passed to Func() to retrieve values // which are stored in the context.Context. This can be useful in tracing, // where span information is commonly propagated in the context.Context. func (e *Event) GetCtx() context.Context { if e == nil || e.ctx == nil { return context.Background() } return e.ctx } // Bool adds the field key with val as a bool to the *Event context. func (e *Event) Bool(key string, b bool) *Event { if e == nil { return e } e.buf = enc.AppendBool(enc.AppendKey(e.buf, key), b) return e } // Bools adds the field key with val as a []bool to the *Event context. func (e *Event) Bools(key string, b []bool) *Event { if e == nil { return e } e.buf = enc.AppendBools(enc.AppendKey(e.buf, key), b) return e } // Int adds the field key with i as a int to the *Event context. func (e *Event) Int(key string, i int) *Event { if e == nil { return e } e.buf = enc.AppendInt(enc.AppendKey(e.buf, key), i) return e } // Ints adds the field key with i as a []int to the *Event context. func (e *Event) Ints(key string, i []int) *Event { if e == nil { return e } e.buf = enc.AppendInts(enc.AppendKey(e.buf, key), i) return e } // Int8 adds the field key with i as a int8 to the *Event context. func (e *Event) Int8(key string, i int8) *Event { if e == nil { return e } e.buf = enc.AppendInt8(enc.AppendKey(e.buf, key), i) return e } // Ints8 adds the field key with i as a []int8 to the *Event context. func (e *Event) Ints8(key string, i []int8) *Event { if e == nil { return e } e.buf = enc.AppendInts8(enc.AppendKey(e.buf, key), i) return e } // Int16 adds the field key with i as a int16 to the *Event context. func (e *Event) Int16(key string, i int16) *Event { if e == nil { return e } e.buf = enc.AppendInt16(enc.AppendKey(e.buf, key), i) return e } // Ints16 adds the field key with i as a []int16 to the *Event context. func (e *Event) Ints16(key string, i []int16) *Event { if e == nil { return e } e.buf = enc.AppendInts16(enc.AppendKey(e.buf, key), i) return e } // Int32 adds the field key with i as a int32 to the *Event context. func (e *Event) Int32(key string, i int32) *Event { if e == nil { return e } e.buf = enc.AppendInt32(enc.AppendKey(e.buf, key), i) return e } // Ints32 adds the field key with i as a []int32 to the *Event context. func (e *Event) Ints32(key string, i []int32) *Event { if e == nil { return e } e.buf = enc.AppendInts32(enc.AppendKey(e.buf, key), i) return e } // Int64 adds the field key with i as a int64 to the *Event context. func (e *Event) Int64(key string, i int64) *Event { if e == nil { return e } e.buf = enc.AppendInt64(enc.AppendKey(e.buf, key), i) return e } // Ints64 adds the field key with i as a []int64 to the *Event context. func (e *Event) Ints64(key string, i []int64) *Event { if e == nil { return e } e.buf = enc.AppendInts64(enc.AppendKey(e.buf, key), i) return e } // Uint adds the field key with i as a uint to the *Event context. func (e *Event) Uint(key string, i uint) *Event { if e == nil { return e } e.buf = enc.AppendUint(enc.AppendKey(e.buf, key), i) return e } // Uints adds the field key with i as a []int to the *Event context. func (e *Event) Uints(key string, i []uint) *Event { if e == nil { return e } e.buf = enc.AppendUints(enc.AppendKey(e.buf, key), i) return e } // Uint8 adds the field key with i as a uint8 to the *Event context. func (e *Event) Uint8(key string, i uint8) *Event { if e == nil { return e } e.buf = enc.AppendUint8(enc.AppendKey(e.buf, key), i) return e } // Uints8 adds the field key with i as a []int8 to the *Event context. func (e *Event) Uints8(key string, i []uint8) *Event { if e == nil { return e } e.buf = enc.AppendUints8(enc.AppendKey(e.buf, key), i) return e } // Uint16 adds the field key with i as a uint16 to the *Event context. func (e *Event) Uint16(key string, i uint16) *Event { if e == nil { return e } e.buf = enc.AppendUint16(enc.AppendKey(e.buf, key), i) return e } // Uints16 adds the field key with i as a []int16 to the *Event context. func (e *Event) Uints16(key string, i []uint16) *Event { if e == nil { return e } e.buf = enc.AppendUints16(enc.AppendKey(e.buf, key), i) return e } // Uint32 adds the field key with i as a uint32 to the *Event context. func (e *Event) Uint32(key string, i uint32) *Event { if e == nil { return e } e.buf = enc.AppendUint32(enc.AppendKey(e.buf, key), i) return e } // Uints32 adds the field key with i as a []int32 to the *Event context. func (e *Event) Uints32(key string, i []uint32) *Event { if e == nil { return e } e.buf = enc.AppendUints32(enc.AppendKey(e.buf, key), i) return e } // Uint64 adds the field key with i as a uint64 to the *Event context. func (e *Event) Uint64(key string, i uint64) *Event { if e == nil { return e } e.buf = enc.AppendUint64(enc.AppendKey(e.buf, key), i) return e } // Uints64 adds the field key with i as a []int64 to the *Event context. func (e *Event) Uints64(key string, i []uint64) *Event { if e == nil { return e } e.buf = enc.AppendUints64(enc.AppendKey(e.buf, key), i) return e } // Float32 adds the field key with f as a float32 to the *Event context. func (e *Event) Float32(key string, f float32) *Event { if e == nil { return e } e.buf = enc.AppendFloat32(enc.AppendKey(e.buf, key), f, FloatingPointPrecision) return e } // Floats32 adds the field key with f as a []float32 to the *Event context. func (e *Event) Floats32(key string, f []float32) *Event { if e == nil { return e } e.buf = enc.AppendFloats32(enc.AppendKey(e.buf, key), f, FloatingPointPrecision) return e } // Float64 adds the field key with f as a float64 to the *Event context. func (e *Event) Float64(key string, f float64) *Event { if e == nil { return e } e.buf = enc.AppendFloat64(enc.AppendKey(e.buf, key), f, FloatingPointPrecision) return e } // Floats64 adds the field key with f as a []float64 to the *Event context. func (e *Event) Floats64(key string, f []float64) *Event { if e == nil { return e } e.buf = enc.AppendFloats64(enc.AppendKey(e.buf, key), f, FloatingPointPrecision) return e } // Timestamp adds the current local time as UNIX timestamp to the *Event context with the "time" key. // To customize the key name, change zerolog.TimestampFieldName. // // NOTE: It won't dedupe the "time" key if the *Event (or *Context) has one // already. func (e *Event) Timestamp() *Event { if e == nil { return e } e.buf = enc.AppendTime(enc.AppendKey(e.buf, TimestampFieldName), TimestampFunc(), TimeFieldFormat) return e } // Time adds the field key with t formatted as string using zerolog.TimeFieldFormat. func (e *Event) Time(key string, t time.Time) *Event { if e == nil { return e } e.buf = enc.AppendTime(enc.AppendKey(e.buf, key), t, TimeFieldFormat) return e } // Times adds the field key with t formatted as string using zerolog.TimeFieldFormat. func (e *Event) Times(key string, t []time.Time) *Event { if e == nil { return e } e.buf = enc.AppendTimes(enc.AppendKey(e.buf, key), t, TimeFieldFormat) return e } // Dur adds the field key with duration d stored as zerolog.DurationFieldUnit. // If zerolog.DurationFieldInteger is true, durations are rendered as integer // instead of float. func (e *Event) Dur(key string, d time.Duration) *Event { if e == nil { return e } e.buf = enc.AppendDuration(enc.AppendKey(e.buf, key), d, DurationFieldUnit, DurationFieldInteger, FloatingPointPrecision) return e } // Durs adds the field key with duration d stored as zerolog.DurationFieldUnit. // If zerolog.DurationFieldInteger is true, durations are rendered as integer // instead of float. func (e *Event) Durs(key string, d []time.Duration) *Event { if e == nil { return e } e.buf = enc.AppendDurations(enc.AppendKey(e.buf, key), d, DurationFieldUnit, DurationFieldInteger, FloatingPointPrecision) return e } // TimeDiff adds the field key with positive duration between time t and start. // If time t is not greater than start, duration will be 0. // Duration format follows the same principle as Dur(). func (e *Event) TimeDiff(key string, t time.Time, start time.Time) *Event { if e == nil { return e } var d time.Duration if t.After(start) { d = t.Sub(start) } e.buf = enc.AppendDuration(enc.AppendKey(e.buf, key), d, DurationFieldUnit, DurationFieldInteger, FloatingPointPrecision) return e } // Any is a wrapper around Event.Interface. func (e *Event) Any(key string, i interface{}) *Event { return e.Interface(key, i) } // Interface adds the field key with i marshaled using reflection. func (e *Event) Interface(key string, i interface{}) *Event { if e == nil { return e } if obj, ok := i.(LogObjectMarshaler); ok { return e.Object(key, obj) } e.buf = enc.AppendInterface(enc.AppendKey(e.buf, key), i) return e } // Type adds the field key with val's type using reflection. func (e *Event) Type(key string, val interface{}) *Event { if e == nil { return e } e.buf = enc.AppendType(enc.AppendKey(e.buf, key), val) return e } // CallerSkipFrame instructs any future Caller calls to skip the specified number of frames. // This includes those added via hooks from the context. func (e *Event) CallerSkipFrame(skip int) *Event { if e == nil { return e } e.skipFrame += skip return e } // Caller adds the file:line of the caller with the zerolog.CallerFieldName key. // The argument skip is the number of stack frames to ascend // Skip If not passed, use the global variable CallerSkipFrameCount func (e *Event) Caller(skip ...int) *Event { sk := CallerSkipFrameCount if len(skip) > 0 { sk = skip[0] + CallerSkipFrameCount } return e.caller(sk) } func (e *Event) caller(skip int) *Event { if e == nil { return e } pc, file, line, ok := runtime.Caller(skip + e.skipFrame) if !ok { return e } e.buf = enc.AppendString(enc.AppendKey(e.buf, CallerFieldName), CallerMarshalFunc(pc, file, line)) return e } // IPAddr adds IPv4 or IPv6 Address to the event func (e *Event) IPAddr(key string, ip net.IP) *Event { if e == nil { return e } e.buf = enc.AppendIPAddr(enc.AppendKey(e.buf, key), ip) return e } // IPPrefix adds IPv4 or IPv6 Prefix (address and mask) to the event func (e *Event) IPPrefix(key string, pfx net.IPNet) *Event { if e == nil { return e } e.buf = enc.AppendIPPrefix(enc.AppendKey(e.buf, key), pfx) return e } // MACAddr adds MAC address to the event func (e *Event) MACAddr(key string, ha net.HardwareAddr) *Event { if e == nil { return e } e.buf = enc.AppendMACAddr(enc.AppendKey(e.buf, key), ha) return e }
package zerolog import ( "encoding/json" "net" "sort" "time" "unsafe" ) func isNilValue(i interface{}) bool { return (*[2]uintptr)(unsafe.Pointer(&i))[1] == 0 } func appendFields(dst []byte, fields interface{}, stack bool) []byte { switch fields := fields.(type) { case []interface{}: if n := len(fields); n&0x1 == 1 { // odd number fields = fields[:n-1] } dst = appendFieldList(dst, fields, stack) case map[string]interface{}: keys := make([]string, 0, len(fields)) for key := range fields { keys = append(keys, key) } sort.Strings(keys) kv := make([]interface{}, 2) for _, key := range keys { kv[0], kv[1] = key, fields[key] dst = appendFieldList(dst, kv, stack) } } return dst } func appendFieldList(dst []byte, kvList []interface{}, stack bool) []byte { for i, n := 0, len(kvList); i < n; i += 2 { key, val := kvList[i], kvList[i+1] if key, ok := key.(string); ok { dst = enc.AppendKey(dst, key) } else { continue } if val, ok := val.(LogObjectMarshaler); ok { e := newEvent(nil, 0) e.buf = e.buf[:0] e.appendObject(val) dst = append(dst, e.buf...) putEvent(e) continue } switch val := val.(type) { case string: dst = enc.AppendString(dst, val) case []byte: dst = enc.AppendBytes(dst, val) case error: switch m := ErrorMarshalFunc(val).(type) { case LogObjectMarshaler: e := newEvent(nil, 0) e.buf = e.buf[:0] e.appendObject(m) dst = append(dst, e.buf...) putEvent(e) case error: if m == nil || isNilValue(m) { dst = enc.AppendNil(dst) } else { dst = enc.AppendString(dst, m.Error()) } case string: dst = enc.AppendString(dst, m) default: dst = enc.AppendInterface(dst, m) } if stack && ErrorStackMarshaler != nil { dst = enc.AppendKey(dst, ErrorStackFieldName) switch m := ErrorStackMarshaler(val).(type) { case nil: case error: if m != nil && !isNilValue(m) { dst = enc.AppendString(dst, m.Error()) } case string: dst = enc.AppendString(dst, m) default: dst = enc.AppendInterface(dst, m) } } case []error: dst = enc.AppendArrayStart(dst) for i, err := range val { switch m := ErrorMarshalFunc(err).(type) { case LogObjectMarshaler: e := newEvent(nil, 0) e.buf = e.buf[:0] e.appendObject(m) dst = append(dst, e.buf...) putEvent(e) case error: if m == nil || isNilValue(m) { dst = enc.AppendNil(dst) } else { dst = enc.AppendString(dst, m.Error()) } case string: dst = enc.AppendString(dst, m) default: dst = enc.AppendInterface(dst, m) } if i < (len(val) - 1) { enc.AppendArrayDelim(dst) } } dst = enc.AppendArrayEnd(dst) case bool: dst = enc.AppendBool(dst, val) case int: dst = enc.AppendInt(dst, val) case int8: dst = enc.AppendInt8(dst, val) case int16: dst = enc.AppendInt16(dst, val) case int32: dst = enc.AppendInt32(dst, val) case int64: dst = enc.AppendInt64(dst, val) case uint: dst = enc.AppendUint(dst, val) case uint8: dst = enc.AppendUint8(dst, val) case uint16: dst = enc.AppendUint16(dst, val) case uint32: dst = enc.AppendUint32(dst, val) case uint64: dst = enc.AppendUint64(dst, val) case float32: dst = enc.AppendFloat32(dst, val, FloatingPointPrecision) case float64: dst = enc.AppendFloat64(dst, val, FloatingPointPrecision) case time.Time: dst = enc.AppendTime(dst, val, TimeFieldFormat) case time.Duration: dst = enc.AppendDuration(dst, val, DurationFieldUnit, DurationFieldInteger, FloatingPointPrecision) case *string: if val != nil { dst = enc.AppendString(dst, *val) } else { dst = enc.AppendNil(dst) } case *bool: if val != nil { dst = enc.AppendBool(dst, *val) } else { dst = enc.AppendNil(dst) } case *int: if val != nil { dst = enc.AppendInt(dst, *val) } else { dst = enc.AppendNil(dst) } case *int8: if val != nil { dst = enc.AppendInt8(dst, *val) } else { dst = enc.AppendNil(dst) } case *int16: if val != nil { dst = enc.AppendInt16(dst, *val) } else { dst = enc.AppendNil(dst) } case *int32: if val != nil { dst = enc.AppendInt32(dst, *val) } else { dst = enc.AppendNil(dst) } case *int64: if val != nil { dst = enc.AppendInt64(dst, *val) } else { dst = enc.AppendNil(dst) } case *uint: if val != nil { dst = enc.AppendUint(dst, *val) } else { dst = enc.AppendNil(dst) } case *uint8: if val != nil { dst = enc.AppendUint8(dst, *val) } else { dst = enc.AppendNil(dst) } case *uint16: if val != nil { dst = enc.AppendUint16(dst, *val) } else { dst = enc.AppendNil(dst) } case *uint32: if val != nil { dst = enc.AppendUint32(dst, *val) } else { dst = enc.AppendNil(dst) } case *uint64: if val != nil { dst = enc.AppendUint64(dst, *val) } else { dst = enc.AppendNil(dst) } case *float32: if val != nil { dst = enc.AppendFloat32(dst, *val, FloatingPointPrecision) } else { dst = enc.AppendNil(dst) } case *float64: if val != nil { dst = enc.AppendFloat64(dst, *val, FloatingPointPrecision) } else { dst = enc.AppendNil(dst) } case *time.Time: if val != nil { dst = enc.AppendTime(dst, *val, TimeFieldFormat) } else { dst = enc.AppendNil(dst) } case *time.Duration: if val != nil { dst = enc.AppendDuration(dst, *val, DurationFieldUnit, DurationFieldInteger, FloatingPointPrecision) } else { dst = enc.AppendNil(dst) } case []string: dst = enc.AppendStrings(dst, val) case []bool: dst = enc.AppendBools(dst, val) case []int: dst = enc.AppendInts(dst, val) case []int8: dst = enc.AppendInts8(dst, val) case []int16: dst = enc.AppendInts16(dst, val) case []int32: dst = enc.AppendInts32(dst, val) case []int64: dst = enc.AppendInts64(dst, val) case []uint: dst = enc.AppendUints(dst, val) // case []uint8: // dst = enc.AppendUints8(dst, val) case []uint16: dst = enc.AppendUints16(dst, val) case []uint32: dst = enc.AppendUints32(dst, val) case []uint64: dst = enc.AppendUints64(dst, val) case []float32: dst = enc.AppendFloats32(dst, val, FloatingPointPrecision) case []float64: dst = enc.AppendFloats64(dst, val, FloatingPointPrecision) case []time.Time: dst = enc.AppendTimes(dst, val, TimeFieldFormat) case []time.Duration: dst = enc.AppendDurations(dst, val, DurationFieldUnit, DurationFieldInteger, FloatingPointPrecision) case nil: dst = enc.AppendNil(dst) case net.IP: dst = enc.AppendIPAddr(dst, val) case net.IPNet: dst = enc.AppendIPPrefix(dst, val) case net.HardwareAddr: dst = enc.AppendMACAddr(dst, val) case json.RawMessage: dst = appendJSON(dst, val) default: dst = enc.AppendInterface(dst, val) } } return dst }
package zerolog import ( "bytes" "encoding/json" "strconv" "sync/atomic" "time" ) const ( // TimeFormatUnix defines a time format that makes time fields to be // serialized as Unix timestamp integers. TimeFormatUnix = "" // TimeFormatUnixMs defines a time format that makes time fields to be // serialized as Unix timestamp integers in milliseconds. TimeFormatUnixMs = "UNIXMS" // TimeFormatUnixMicro defines a time format that makes time fields to be // serialized as Unix timestamp integers in microseconds. TimeFormatUnixMicro = "UNIXMICRO" // TimeFormatUnixNano defines a time format that makes time fields to be // serialized as Unix timestamp integers in nanoseconds. TimeFormatUnixNano = "UNIXNANO" ) var ( // TimestampFieldName is the field name used for the timestamp field. TimestampFieldName = "time" // LevelFieldName is the field name used for the level field. LevelFieldName = "level" // LevelTraceValue is the value used for the trace level field. LevelTraceValue = "trace" // LevelDebugValue is the value used for the debug level field. LevelDebugValue = "debug" // LevelInfoValue is the value used for the info level field. LevelInfoValue = "info" // LevelWarnValue is the value used for the warn level field. LevelWarnValue = "warn" // LevelErrorValue is the value used for the error level field. LevelErrorValue = "error" // LevelFatalValue is the value used for the fatal level field. LevelFatalValue = "fatal" // LevelPanicValue is the value used for the panic level field. LevelPanicValue = "panic" // LevelFieldMarshalFunc allows customization of global level field marshaling. LevelFieldMarshalFunc = func(l Level) string { return l.String() } // MessageFieldName is the field name used for the message field. MessageFieldName = "message" // ErrorFieldName is the field name used for error fields. ErrorFieldName = "error" // CallerFieldName is the field name used for caller field. CallerFieldName = "caller" // CallerSkipFrameCount is the number of stack frames to skip to find the caller. CallerSkipFrameCount = 2 // CallerMarshalFunc allows customization of global caller marshaling CallerMarshalFunc = func(pc uintptr, file string, line int) string { return file + ":" + strconv.Itoa(line) } // ErrorStackFieldName is the field name used for error stacks. ErrorStackFieldName = "stack" // ErrorStackMarshaler extract the stack from err if any. ErrorStackMarshaler func(err error) interface{} // ErrorMarshalFunc allows customization of global error marshaling ErrorMarshalFunc = func(err error) interface{} { return err } // InterfaceMarshalFunc allows customization of interface marshaling. // Default: "encoding/json.Marshal" with disabled HTML escaping InterfaceMarshalFunc = func(v interface{}) ([]byte, error) { var buf bytes.Buffer encoder := json.NewEncoder(&buf) encoder.SetEscapeHTML(false) err := encoder.Encode(v) if err != nil { return nil, err } b := buf.Bytes() if len(b) > 0 { // Remove trailing \n which is added by Encode. return b[:len(b)-1], nil } return b, nil } // TimeFieldFormat defines the time format of the Time field type. If set to // TimeFormatUnix, TimeFormatUnixMs, TimeFormatUnixMicro or TimeFormatUnixNano, the time is formatted as a UNIX // timestamp as integer. TimeFieldFormat = time.RFC3339 // TimestampFunc defines the function called to generate a timestamp. TimestampFunc = time.Now // DurationFieldUnit defines the unit for time.Duration type fields added // using the Dur method. DurationFieldUnit = time.Millisecond // DurationFieldInteger renders Dur fields as integer instead of float if // set to true. DurationFieldInteger = false // ErrorHandler is called whenever zerolog fails to write an event on its // output. If not set, an error is printed on the stderr. This handler must // be thread safe and non-blocking. ErrorHandler func(err error) // DefaultContextLogger is returned from Ctx() if there is no logger associated // with the context. DefaultContextLogger *Logger // LevelColors are used by ConsoleWriter's consoleDefaultFormatLevel to color // log levels. LevelColors = map[Level]int{ TraceLevel: colorBlue, DebugLevel: 0, InfoLevel: colorGreen, WarnLevel: colorYellow, ErrorLevel: colorRed, FatalLevel: colorRed, PanicLevel: colorRed, } // FormattedLevels are used by ConsoleWriter's consoleDefaultFormatLevel // for a short level name. FormattedLevels = map[Level]string{ TraceLevel: "TRC", DebugLevel: "DBG", InfoLevel: "INF", WarnLevel: "WRN", ErrorLevel: "ERR", FatalLevel: "FTL", PanicLevel: "PNC", } // TriggerLevelWriterBufferReuseLimit is a limit in bytes that a buffer is dropped // from the TriggerLevelWriter buffer pool if the buffer grows above the limit. TriggerLevelWriterBufferReuseLimit = 64 * 1024 // FloatingPointPrecision, if set to a value other than -1, controls the number // of digits when formatting float numbers in JSON. See strconv.FormatFloat for // more details. FloatingPointPrecision = -1 ) var ( gLevel = new(int32) disableSampling = new(int32) ) // SetGlobalLevel sets the global override for log level. If this // values is raised, all Loggers will use at least this value. // // To globally disable logs, set GlobalLevel to Disabled. func SetGlobalLevel(l Level) { atomic.StoreInt32(gLevel, int32(l)) } // GlobalLevel returns the current global log level func GlobalLevel() Level { return Level(atomic.LoadInt32(gLevel)) } // DisableSampling will disable sampling in all Loggers if true. func DisableSampling(v bool) { var i int32 if v { i = 1 } atomic.StoreInt32(disableSampling, i) } func samplingDisabled() bool { return atomic.LoadInt32(disableSampling) == 1 }
// Package hlog provides a set of http.Handler helpers for zerolog. package hlog import ( "context" "net" "net/http" "strings" "time" "github.com/rs/xid" "github.com/rs/zerolog" "github.com/rs/zerolog/hlog/internal/mutil" "github.com/rs/zerolog/log" ) // FromRequest gets the logger in the request's context. // This is a shortcut for log.Ctx(r.Context()) func FromRequest(r *http.Request) *zerolog.Logger { return log.Ctx(r.Context()) } // NewHandler injects log into requests context. func NewHandler(log zerolog.Logger) func(http.Handler) http.Handler { return func(next http.Handler) http.Handler { return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { // Create a copy of the logger (including internal context slice) // to prevent data race when using UpdateContext. l := log.With().Logger() r = r.WithContext(l.WithContext(r.Context())) next.ServeHTTP(w, r) }) } } // URLHandler adds the requested URL as a field to the context's logger // using fieldKey as field key. func URLHandler(fieldKey string) func(next http.Handler) http.Handler { return func(next http.Handler) http.Handler { return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { log := zerolog.Ctx(r.Context()) log.UpdateContext(func(c zerolog.Context) zerolog.Context { return c.Str(fieldKey, r.URL.String()) }) next.ServeHTTP(w, r) }) } } // MethodHandler adds the request method as a field to the context's logger // using fieldKey as field key. func MethodHandler(fieldKey string) func(next http.Handler) http.Handler { return func(next http.Handler) http.Handler { return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { log := zerolog.Ctx(r.Context()) log.UpdateContext(func(c zerolog.Context) zerolog.Context { return c.Str(fieldKey, r.Method) }) next.ServeHTTP(w, r) }) } } // RequestHandler adds the request method and URL as a field to the context's logger // using fieldKey as field key. func RequestHandler(fieldKey string) func(next http.Handler) http.Handler { return func(next http.Handler) http.Handler { return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { log := zerolog.Ctx(r.Context()) log.UpdateContext(func(c zerolog.Context) zerolog.Context { return c.Str(fieldKey, r.Method+" "+r.URL.String()) }) next.ServeHTTP(w, r) }) } } // RemoteAddrHandler adds the request's remote address as a field to the context's logger // using fieldKey as field key. func RemoteAddrHandler(fieldKey string) func(next http.Handler) http.Handler { return func(next http.Handler) http.Handler { return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { if r.RemoteAddr != "" { log := zerolog.Ctx(r.Context()) log.UpdateContext(func(c zerolog.Context) zerolog.Context { return c.Str(fieldKey, r.RemoteAddr) }) } next.ServeHTTP(w, r) }) } } func getHost(hostPort string) string { if hostPort == "" { return "" } host, _, err := net.SplitHostPort(hostPort) if err != nil { return hostPort } return host } // RemoteIPHandler is similar to RemoteAddrHandler, but logs only // an IP, not a port. func RemoteIPHandler(fieldKey string) func(next http.Handler) http.Handler { return func(next http.Handler) http.Handler { return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { ip := getHost(r.RemoteAddr) if ip != "" { log := zerolog.Ctx(r.Context()) log.UpdateContext(func(c zerolog.Context) zerolog.Context { return c.Str(fieldKey, ip) }) } next.ServeHTTP(w, r) }) } } // UserAgentHandler adds the request's user-agent as a field to the context's logger // using fieldKey as field key. func UserAgentHandler(fieldKey string) func(next http.Handler) http.Handler { return func(next http.Handler) http.Handler { return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { if ua := r.Header.Get("User-Agent"); ua != "" { log := zerolog.Ctx(r.Context()) log.UpdateContext(func(c zerolog.Context) zerolog.Context { return c.Str(fieldKey, ua) }) } next.ServeHTTP(w, r) }) } } // RefererHandler adds the request's referer as a field to the context's logger // using fieldKey as field key. func RefererHandler(fieldKey string) func(next http.Handler) http.Handler { return func(next http.Handler) http.Handler { return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { if ref := r.Header.Get("Referer"); ref != "" { log := zerolog.Ctx(r.Context()) log.UpdateContext(func(c zerolog.Context) zerolog.Context { return c.Str(fieldKey, ref) }) } next.ServeHTTP(w, r) }) } } // ProtoHandler adds the requests protocol version as a field to the context logger // using fieldKey as field Key. func ProtoHandler(fieldKey string) func(next http.Handler) http.Handler { return func(next http.Handler) http.Handler { return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { log := zerolog.Ctx(r.Context()) log.UpdateContext(func(c zerolog.Context) zerolog.Context { return c.Str(fieldKey, r.Proto) }) next.ServeHTTP(w, r) }) } } // HTTPVersionHandler is similar to ProtoHandler, but it does not store the "HTTP/" // prefix in the protocol name. func HTTPVersionHandler(fieldKey string) func(next http.Handler) http.Handler { return func(next http.Handler) http.Handler { return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { proto := strings.TrimPrefix(r.Proto, "HTTP/") log := zerolog.Ctx(r.Context()) log.UpdateContext(func(c zerolog.Context) zerolog.Context { return c.Str(fieldKey, proto) }) next.ServeHTTP(w, r) }) } } type idKey struct{} // IDFromRequest returns the unique id associated to the request if any. func IDFromRequest(r *http.Request) (id xid.ID, ok bool) { if r == nil { return } return IDFromCtx(r.Context()) } // IDFromCtx returns the unique id associated to the context if any. func IDFromCtx(ctx context.Context) (id xid.ID, ok bool) { id, ok = ctx.Value(idKey{}).(xid.ID) return } // CtxWithID adds the given xid.ID to the context func CtxWithID(ctx context.Context, id xid.ID) context.Context { return context.WithValue(ctx, idKey{}, id) } // RequestIDHandler returns a handler setting a unique id to the request which can // be gathered using IDFromRequest(req). This generated id is added as a field to the // logger using the passed fieldKey as field name. The id is also added as a response // header if the headerName is not empty. // // The generated id is a URL safe base64 encoded mongo object-id-like unique id. // Mongo unique id generation algorithm has been selected as a trade-off between // size and ease of use: UUID is less space efficient and snowflake requires machine // configuration. func RequestIDHandler(fieldKey, headerName string) func(next http.Handler) http.Handler { return func(next http.Handler) http.Handler { return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { ctx := r.Context() id, ok := IDFromRequest(r) if !ok { id = xid.New() ctx = CtxWithID(ctx, id) r = r.WithContext(ctx) } if fieldKey != "" { log := zerolog.Ctx(ctx) log.UpdateContext(func(c zerolog.Context) zerolog.Context { return c.Str(fieldKey, id.String()) }) } if headerName != "" { w.Header().Set(headerName, id.String()) } next.ServeHTTP(w, r) }) } } // CustomHeaderHandler adds given header from request's header as a field to // the context's logger using fieldKey as field key. func CustomHeaderHandler(fieldKey, header string) func(next http.Handler) http.Handler { return func(next http.Handler) http.Handler { return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { if val := r.Header.Get(header); val != "" { log := zerolog.Ctx(r.Context()) log.UpdateContext(func(c zerolog.Context) zerolog.Context { return c.Str(fieldKey, val) }) } next.ServeHTTP(w, r) }) } } // EtagHandler adds Etag header from response's header as a field to // the context's logger using fieldKey as field key. func EtagHandler(fieldKey string) func(next http.Handler) http.Handler { return func(next http.Handler) http.Handler { return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { defer func() { etag := w.Header().Get("Etag") if etag != "" { etag = strings.ReplaceAll(etag, `"`, "") log := zerolog.Ctx(r.Context()) log.UpdateContext(func(c zerolog.Context) zerolog.Context { return c.Str(fieldKey, etag) }) } }() next.ServeHTTP(w, r) }) } } func ResponseHeaderHandler(fieldKey, headerName string) func(next http.Handler) http.Handler { return func(next http.Handler) http.Handler { return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { defer func() { value := w.Header().Get(headerName) if value != "" { log := zerolog.Ctx(r.Context()) log.UpdateContext(func(c zerolog.Context) zerolog.Context { return c.Str(fieldKey, value) }) } }() next.ServeHTTP(w, r) }) } } // AccessHandler returns a handler that call f after each request. func AccessHandler(f func(r *http.Request, status, size int, duration time.Duration)) func(next http.Handler) http.Handler { return func(next http.Handler) http.Handler { return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { start := time.Now() lw := mutil.WrapWriter(w) defer func() { f(r, lw.Status(), lw.BytesWritten(), time.Since(start)) }() next.ServeHTTP(lw, r) }) } } // HostHandler adds the request's host as a field to the context's logger // using fieldKey as field key. If trimPort is set to true, then port is // removed from the host. func HostHandler(fieldKey string, trimPort ...bool) func(next http.Handler) http.Handler { return func(next http.Handler) http.Handler { return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { var host string if len(trimPort) > 0 && trimPort[0] { host = getHost(r.Host) } else { host = r.Host } if host != "" { log := zerolog.Ctx(r.Context()) log.UpdateContext(func(c zerolog.Context) zerolog.Context { return c.Str(fieldKey, host) }) } next.ServeHTTP(w, r) }) } }
package mutil import ( "bufio" "io" "net" "net/http" ) // WriterProxy is a proxy around an http.ResponseWriter that allows you to hook // into various parts of the response process. type WriterProxy interface { http.ResponseWriter // Status returns the HTTP status of the request, or 0 if one has not // yet been sent. Status() int // BytesWritten returns the total number of bytes sent to the client. BytesWritten() int // Tee causes the response body to be written to the given io.Writer in // addition to proxying the writes through. Only one io.Writer can be // tee'd to at once: setting a second one will overwrite the first. // Writes will be sent to the proxy before being written to this // io.Writer. It is illegal for the tee'd writer to be modified // concurrently with writes. Tee(io.Writer) // Unwrap returns the original proxied target. Unwrap() http.ResponseWriter } // WrapWriter wraps an http.ResponseWriter, returning a proxy that allows you to // hook into various parts of the response process. func WrapWriter(w http.ResponseWriter) WriterProxy { _, cn := w.(http.CloseNotifier) _, fl := w.(http.Flusher) _, hj := w.(http.Hijacker) _, rf := w.(io.ReaderFrom) bw := basicWriter{ResponseWriter: w} if cn && fl && hj && rf { return &fancyWriter{bw} } if fl { return &flushWriter{bw} } return &bw } // basicWriter wraps a http.ResponseWriter that implements the minimal // http.ResponseWriter interface. type basicWriter struct { http.ResponseWriter wroteHeader bool code int bytes int tee io.Writer } func (b *basicWriter) WriteHeader(code int) { if !b.wroteHeader { b.code = code b.wroteHeader = true b.ResponseWriter.WriteHeader(code) } } func (b *basicWriter) Write(buf []byte) (int, error) { b.WriteHeader(http.StatusOK) n, err := b.ResponseWriter.Write(buf) if b.tee != nil { _, err2 := b.tee.Write(buf[:n]) // Prefer errors generated by the proxied writer. if err == nil { err = err2 } } b.bytes += n return n, err } func (b *basicWriter) maybeWriteHeader() { if !b.wroteHeader { b.WriteHeader(http.StatusOK) } } func (b *basicWriter) Status() int { return b.code } func (b *basicWriter) BytesWritten() int { return b.bytes } func (b *basicWriter) Tee(w io.Writer) { b.tee = w } func (b *basicWriter) Unwrap() http.ResponseWriter { return b.ResponseWriter } // fancyWriter is a writer that additionally satisfies http.CloseNotifier, // http.Flusher, http.Hijacker, and io.ReaderFrom. It exists for the common case // of wrapping the http.ResponseWriter that package http gives you, in order to // make the proxied object support the full method set of the proxied object. type fancyWriter struct { basicWriter } func (f *fancyWriter) CloseNotify() <-chan bool { cn := f.basicWriter.ResponseWriter.(http.CloseNotifier) return cn.CloseNotify() } func (f *fancyWriter) Flush() { fl := f.basicWriter.ResponseWriter.(http.Flusher) fl.Flush() } func (f *fancyWriter) Hijack() (net.Conn, *bufio.ReadWriter, error) { hj := f.basicWriter.ResponseWriter.(http.Hijacker) return hj.Hijack() } func (f *fancyWriter) ReadFrom(r io.Reader) (int64, error) { if f.basicWriter.tee != nil { n, err := io.Copy(&f.basicWriter, r) f.bytes += int(n) return n, err } rf := f.basicWriter.ResponseWriter.(io.ReaderFrom) f.basicWriter.maybeWriteHeader() n, err := rf.ReadFrom(r) f.bytes += int(n) return n, err } type flushWriter struct { basicWriter } func (f *flushWriter) Flush() { fl := f.basicWriter.ResponseWriter.(http.Flusher) fl.Flush() } var ( _ http.CloseNotifier = &fancyWriter{} _ http.Flusher = &fancyWriter{} _ http.Hijacker = &fancyWriter{} _ io.ReaderFrom = &fancyWriter{} _ http.Flusher = &flushWriter{} )
package zerolog // Hook defines an interface to a log hook. type Hook interface { // Run runs the hook with the event. Run(e *Event, level Level, message string) } // HookFunc is an adaptor to allow the use of an ordinary function // as a Hook. type HookFunc func(e *Event, level Level, message string) // Run implements the Hook interface. func (h HookFunc) Run(e *Event, level Level, message string) { h(e, level, message) } // LevelHook applies a different hook for each level. type LevelHook struct { NoLevelHook, TraceHook, DebugHook, InfoHook, WarnHook, ErrorHook, FatalHook, PanicHook Hook } // Run implements the Hook interface. func (h LevelHook) Run(e *Event, level Level, message string) { switch level { case TraceLevel: if h.TraceHook != nil { h.TraceHook.Run(e, level, message) } case DebugLevel: if h.DebugHook != nil { h.DebugHook.Run(e, level, message) } case InfoLevel: if h.InfoHook != nil { h.InfoHook.Run(e, level, message) } case WarnLevel: if h.WarnHook != nil { h.WarnHook.Run(e, level, message) } case ErrorLevel: if h.ErrorHook != nil { h.ErrorHook.Run(e, level, message) } case FatalLevel: if h.FatalHook != nil { h.FatalHook.Run(e, level, message) } case PanicLevel: if h.PanicHook != nil { h.PanicHook.Run(e, level, message) } case NoLevel: if h.NoLevelHook != nil { h.NoLevelHook.Run(e, level, message) } } } // NewLevelHook returns a new LevelHook. func NewLevelHook() LevelHook { return LevelHook{} }
package cbor // JSONMarshalFunc is used to marshal interface to JSON encoded byte slice. // Making it package level instead of embedded in Encoder brings // some extra efforts at importing, but avoids value copy when the functions // of Encoder being invoked. // DO REMEMBER to set this variable at importing, or // you might get a nil pointer dereference panic at runtime. var JSONMarshalFunc func(v interface{}) ([]byte, error) type Encoder struct{} // AppendKey adds a key (string) to the binary encoded log message func (e Encoder) AppendKey(dst []byte, key string) []byte { if len(dst) < 1 { dst = e.AppendBeginMarker(dst) } return e.AppendString(dst, key) }
// Package cbor provides primitives for storing different data // in the CBOR (binary) format. CBOR is defined in RFC7049. package cbor import "time" const ( majorOffset = 5 additionalMax = 23 // Non Values. additionalTypeBoolFalse byte = 20 additionalTypeBoolTrue byte = 21 additionalTypeNull byte = 22 // Integer (+ve and -ve) Sub-types. additionalTypeIntUint8 byte = 24 additionalTypeIntUint16 byte = 25 additionalTypeIntUint32 byte = 26 additionalTypeIntUint64 byte = 27 // Float Sub-types. additionalTypeFloat16 byte = 25 additionalTypeFloat32 byte = 26 additionalTypeFloat64 byte = 27 additionalTypeBreak byte = 31 // Tag Sub-types. additionalTypeTimestamp byte = 01 additionalTypeEmbeddedCBOR byte = 63 // Extended Tags - from https://www.iana.org/assignments/cbor-tags/cbor-tags.xhtml additionalTypeTagNetworkAddr uint16 = 260 additionalTypeTagNetworkPrefix uint16 = 261 additionalTypeEmbeddedJSON uint16 = 262 additionalTypeTagHexString uint16 = 263 // Unspecified number of elements. additionalTypeInfiniteCount byte = 31 ) const ( majorTypeUnsignedInt byte = iota << majorOffset // Major type 0 majorTypeNegativeInt // Major type 1 majorTypeByteString // Major type 2 majorTypeUtf8String // Major type 3 majorTypeArray // Major type 4 majorTypeMap // Major type 5 majorTypeTags // Major type 6 majorTypeSimpleAndFloat // Major type 7 ) const ( maskOutAdditionalType byte = (7 << majorOffset) maskOutMajorType byte = 31 ) const ( float32Nan = "\xfa\x7f\xc0\x00\x00" float32PosInfinity = "\xfa\x7f\x80\x00\x00" float32NegInfinity = "\xfa\xff\x80\x00\x00" float64Nan = "\xfb\x7f\xf8\x00\x00\x00\x00\x00\x00" float64PosInfinity = "\xfb\x7f\xf0\x00\x00\x00\x00\x00\x00" float64NegInfinity = "\xfb\xff\xf0\x00\x00\x00\x00\x00\x00" ) // IntegerTimeFieldFormat indicates the format of timestamp decoded // from an integer (time in seconds). var IntegerTimeFieldFormat = time.RFC3339 // NanoTimeFieldFormat indicates the format of timestamp decoded // from a float value (time in seconds and nanoseconds). var NanoTimeFieldFormat = time.RFC3339Nano func appendCborTypePrefix(dst []byte, major byte, number uint64) []byte { byteCount := 8 var minor byte switch { case number < 256: byteCount = 1 minor = additionalTypeIntUint8 case number < 65536: byteCount = 2 minor = additionalTypeIntUint16 case number < 4294967296: byteCount = 4 minor = additionalTypeIntUint32 default: byteCount = 8 minor = additionalTypeIntUint64 } dst = append(dst, major|minor) byteCount-- for ; byteCount >= 0; byteCount-- { dst = append(dst, byte(number>>(uint(byteCount)*8))) } return dst }
package cbor // This file contains code to decode a stream of CBOR Data into JSON. import ( "bufio" "bytes" "encoding/base64" "fmt" "io" "math" "net" "runtime" "strconv" "strings" "time" "unicode/utf8" ) var decodeTimeZone *time.Location const hexTable = "0123456789abcdef" const isFloat32 = 4 const isFloat64 = 8 func readNBytes(src *bufio.Reader, n int) []byte { ret := make([]byte, n) for i := 0; i < n; i++ { ch, e := src.ReadByte() if e != nil { panic(fmt.Errorf("Tried to Read %d Bytes.. But hit end of file", n)) } ret[i] = ch } return ret } func readByte(src *bufio.Reader) byte { b, e := src.ReadByte() if e != nil { panic(fmt.Errorf("Tried to Read 1 Byte.. But hit end of file")) } return b } func decodeIntAdditionalType(src *bufio.Reader, minor byte) int64 { val := int64(0) if minor <= 23 { val = int64(minor) } else { bytesToRead := 0 switch minor { case additionalTypeIntUint8: bytesToRead = 1 case additionalTypeIntUint16: bytesToRead = 2 case additionalTypeIntUint32: bytesToRead = 4 case additionalTypeIntUint64: bytesToRead = 8 default: panic(fmt.Errorf("Invalid Additional Type: %d in decodeInteger (expected <28)", minor)) } pb := readNBytes(src, bytesToRead) for i := 0; i < bytesToRead; i++ { val = val * 256 val += int64(pb[i]) } } return val } func decodeInteger(src *bufio.Reader) int64 { pb := readByte(src) major := pb & maskOutAdditionalType minor := pb & maskOutMajorType if major != majorTypeUnsignedInt && major != majorTypeNegativeInt { panic(fmt.Errorf("Major type is: %d in decodeInteger!! (expected 0 or 1)", major)) } val := decodeIntAdditionalType(src, minor) if major == 0 { return val } return (-1 - val) } func decodeFloat(src *bufio.Reader) (float64, int) { pb := readByte(src) major := pb & maskOutAdditionalType minor := pb & maskOutMajorType if major != majorTypeSimpleAndFloat { panic(fmt.Errorf("Incorrect Major type is: %d in decodeFloat", major)) } switch minor { case additionalTypeFloat16: panic(fmt.Errorf("float16 is not supported in decodeFloat")) case additionalTypeFloat32: pb := readNBytes(src, 4) switch string(pb) { case float32Nan: return math.NaN(), isFloat32 case float32PosInfinity: return math.Inf(0), isFloat32 case float32NegInfinity: return math.Inf(-1), isFloat32 } n := uint32(0) for i := 0; i < 4; i++ { n = n * 256 n += uint32(pb[i]) } val := math.Float32frombits(n) return float64(val), isFloat32 case additionalTypeFloat64: pb := readNBytes(src, 8) switch string(pb) { case float64Nan: return math.NaN(), isFloat64 case float64PosInfinity: return math.Inf(0), isFloat64 case float64NegInfinity: return math.Inf(-1), isFloat64 } n := uint64(0) for i := 0; i < 8; i++ { n = n * 256 n += uint64(pb[i]) } val := math.Float64frombits(n) return val, isFloat64 } panic(fmt.Errorf("Invalid Additional Type: %d in decodeFloat", minor)) } func decodeStringComplex(dst []byte, s string, pos uint) []byte { i := int(pos) start := 0 for i < len(s) { b := s[i] if b >= utf8.RuneSelf { r, size := utf8.DecodeRuneInString(s[i:]) if r == utf8.RuneError && size == 1 { // In case of error, first append previous simple characters to // the byte slice if any and append a replacement character code // in place of the invalid sequence. if start < i { dst = append(dst, s[start:i]...) } dst = append(dst, `\ufffd`...) i += size start = i continue } i += size continue } if b >= 0x20 && b <= 0x7e && b != '\\' && b != '"' { i++ continue } // We encountered a character that needs to be encoded. // Let's append the previous simple characters to the byte slice // and switch our operation to read and encode the remainder // characters byte-by-byte. if start < i { dst = append(dst, s[start:i]...) } switch b { case '"', '\\': dst = append(dst, '\\', b) case '\b': dst = append(dst, '\\', 'b') case '\f': dst = append(dst, '\\', 'f') case '\n': dst = append(dst, '\\', 'n') case '\r': dst = append(dst, '\\', 'r') case '\t': dst = append(dst, '\\', 't') default: dst = append(dst, '\\', 'u', '0', '0', hexTable[b>>4], hexTable[b&0xF]) } i++ start = i } if start < len(s) { dst = append(dst, s[start:]...) } return dst } func decodeString(src *bufio.Reader, noQuotes bool) []byte { pb := readByte(src) major := pb & maskOutAdditionalType minor := pb & maskOutMajorType if major != majorTypeByteString { panic(fmt.Errorf("Major type is: %d in decodeString", major)) } result := []byte{} if !noQuotes { result = append(result, '"') } length := decodeIntAdditionalType(src, minor) len := int(length) pbs := readNBytes(src, len) result = append(result, pbs...) if noQuotes { return result } return append(result, '"') } func decodeStringToDataUrl(src *bufio.Reader, mimeType string) []byte { pb := readByte(src) major := pb & maskOutAdditionalType minor := pb & maskOutMajorType if major != majorTypeByteString { panic(fmt.Errorf("Major type is: %d in decodeString", major)) } length := decodeIntAdditionalType(src, minor) l := int(length) enc := base64.StdEncoding lEnc := enc.EncodedLen(l) result := make([]byte, len("\"data:;base64,\"")+len(mimeType)+lEnc) dest := result u := copy(dest, "\"data:") dest = dest[u:] u = copy(dest, mimeType) dest = dest[u:] u = copy(dest, ";base64,") dest = dest[u:] pbs := readNBytes(src, l) enc.Encode(dest, pbs) dest = dest[lEnc:] dest[0] = '"' return result } func decodeUTF8String(src *bufio.Reader) []byte { pb := readByte(src) major := pb & maskOutAdditionalType minor := pb & maskOutMajorType if major != majorTypeUtf8String { panic(fmt.Errorf("Major type is: %d in decodeUTF8String", major)) } result := []byte{'"'} length := decodeIntAdditionalType(src, minor) len := int(length) pbs := readNBytes(src, len) for i := 0; i < len; i++ { // Check if the character needs encoding. Control characters, slashes, // and the double quote need json encoding. Bytes above the ascii // boundary needs utf8 encoding. if pbs[i] < 0x20 || pbs[i] > 0x7e || pbs[i] == '\\' || pbs[i] == '"' { // We encountered a character that needs to be encoded. Switch // to complex version of the algorithm. dst := []byte{'"'} dst = decodeStringComplex(dst, string(pbs), uint(i)) return append(dst, '"') } } // The string has no need for encoding and therefore is directly // appended to the byte slice. result = append(result, pbs...) return append(result, '"') } func array2Json(src *bufio.Reader, dst io.Writer) { dst.Write([]byte{'['}) pb := readByte(src) major := pb & maskOutAdditionalType minor := pb & maskOutMajorType if major != majorTypeArray { panic(fmt.Errorf("Major type is: %d in array2Json", major)) } len := 0 unSpecifiedCount := false if minor == additionalTypeInfiniteCount { unSpecifiedCount = true } else { length := decodeIntAdditionalType(src, minor) len = int(length) } for i := 0; unSpecifiedCount || i < len; i++ { if unSpecifiedCount { pb, e := src.Peek(1) if e != nil { panic(e) } if pb[0] == majorTypeSimpleAndFloat|additionalTypeBreak { readByte(src) break } } cbor2JsonOneObject(src, dst) if unSpecifiedCount { pb, e := src.Peek(1) if e != nil { panic(e) } if pb[0] == majorTypeSimpleAndFloat|additionalTypeBreak { readByte(src) break } dst.Write([]byte{','}) } else if i+1 < len { dst.Write([]byte{','}) } } dst.Write([]byte{']'}) } func map2Json(src *bufio.Reader, dst io.Writer) { pb := readByte(src) major := pb & maskOutAdditionalType minor := pb & maskOutMajorType if major != majorTypeMap { panic(fmt.Errorf("Major type is: %d in map2Json", major)) } len := 0 unSpecifiedCount := false if minor == additionalTypeInfiniteCount { unSpecifiedCount = true } else { length := decodeIntAdditionalType(src, minor) len = int(length) } dst.Write([]byte{'{'}) for i := 0; unSpecifiedCount || i < len; i++ { if unSpecifiedCount { pb, e := src.Peek(1) if e != nil { panic(e) } if pb[0] == majorTypeSimpleAndFloat|additionalTypeBreak { readByte(src) break } } cbor2JsonOneObject(src, dst) if i%2 == 0 { // Even position values are keys. dst.Write([]byte{':'}) } else { if unSpecifiedCount { pb, e := src.Peek(1) if e != nil { panic(e) } if pb[0] == majorTypeSimpleAndFloat|additionalTypeBreak { readByte(src) break } dst.Write([]byte{','}) } else if i+1 < len { dst.Write([]byte{','}) } } } dst.Write([]byte{'}'}) } func decodeTagData(src *bufio.Reader) []byte { pb := readByte(src) major := pb & maskOutAdditionalType minor := pb & maskOutMajorType if major != majorTypeTags { panic(fmt.Errorf("Major type is: %d in decodeTagData", major)) } switch minor { case additionalTypeTimestamp: return decodeTimeStamp(src) case additionalTypeIntUint8: val := decodeIntAdditionalType(src, minor) switch byte(val) { case additionalTypeEmbeddedCBOR: pb := readByte(src) dataMajor := pb & maskOutAdditionalType if dataMajor != majorTypeByteString { panic(fmt.Errorf("Unsupported embedded Type: %d in decodeEmbeddedCBOR", dataMajor)) } src.UnreadByte() return decodeStringToDataUrl(src, "application/cbor") default: panic(fmt.Errorf("Unsupported Additional Tag Type: %d in decodeTagData", val)) } // Tag value is larger than 256 (so uint16). case additionalTypeIntUint16: val := decodeIntAdditionalType(src, minor) switch uint16(val) { case additionalTypeEmbeddedJSON: pb := readByte(src) dataMajor := pb & maskOutAdditionalType if dataMajor != majorTypeByteString { panic(fmt.Errorf("Unsupported embedded Type: %d in decodeEmbeddedJSON", dataMajor)) } src.UnreadByte() return decodeString(src, true) case additionalTypeTagNetworkAddr: octets := decodeString(src, true) ss := []byte{'"'} switch len(octets) { case 6: // MAC address. ha := net.HardwareAddr(octets) ss = append(append(ss, ha.String()...), '"') case 4: // IPv4 address. fallthrough case 16: // IPv6 address. ip := net.IP(octets) ss = append(append(ss, ip.String()...), '"') default: panic(fmt.Errorf("Unexpected Network Address length: %d (expected 4,6,16)", len(octets))) } return ss case additionalTypeTagNetworkPrefix: pb := readByte(src) if pb != majorTypeMap|0x1 { panic(fmt.Errorf("IP Prefix is NOT of MAP of 1 elements as expected")) } octets := decodeString(src, true) val := decodeInteger(src) ip := net.IP(octets) var mask net.IPMask pfxLen := int(val) if len(octets) == 4 { mask = net.CIDRMask(pfxLen, 32) } else { mask = net.CIDRMask(pfxLen, 128) } ipPfx := net.IPNet{IP: ip, Mask: mask} ss := []byte{'"'} ss = append(append(ss, ipPfx.String()...), '"') return ss case additionalTypeTagHexString: octets := decodeString(src, true) ss := []byte{'"'} for _, v := range octets { ss = append(ss, hexTable[v>>4], hexTable[v&0x0f]) } return append(ss, '"') default: panic(fmt.Errorf("Unsupported Additional Tag Type: %d in decodeTagData", val)) } } panic(fmt.Errorf("Unsupported Additional Type: %d in decodeTagData", minor)) } func decodeTimeStamp(src *bufio.Reader) []byte { pb := readByte(src) src.UnreadByte() tsMajor := pb & maskOutAdditionalType if tsMajor == majorTypeUnsignedInt || tsMajor == majorTypeNegativeInt { n := decodeInteger(src) t := time.Unix(n, 0) if decodeTimeZone != nil { t = t.In(decodeTimeZone) } else { t = t.In(time.UTC) } tsb := []byte{} tsb = append(tsb, '"') tsb = t.AppendFormat(tsb, IntegerTimeFieldFormat) tsb = append(tsb, '"') return tsb } else if tsMajor == majorTypeSimpleAndFloat { n, _ := decodeFloat(src) secs := int64(n) n -= float64(secs) n *= float64(1e9) t := time.Unix(secs, int64(n)) if decodeTimeZone != nil { t = t.In(decodeTimeZone) } else { t = t.In(time.UTC) } tsb := []byte{} tsb = append(tsb, '"') tsb = t.AppendFormat(tsb, NanoTimeFieldFormat) tsb = append(tsb, '"') return tsb } panic(fmt.Errorf("TS format is neigther int nor float: %d", tsMajor)) } func decodeSimpleFloat(src *bufio.Reader) []byte { pb := readByte(src) major := pb & maskOutAdditionalType minor := pb & maskOutMajorType if major != majorTypeSimpleAndFloat { panic(fmt.Errorf("Major type is: %d in decodeSimpleFloat", major)) } switch minor { case additionalTypeBoolTrue: return []byte("true") case additionalTypeBoolFalse: return []byte("false") case additionalTypeNull: return []byte("null") case additionalTypeFloat16: fallthrough case additionalTypeFloat32: fallthrough case additionalTypeFloat64: src.UnreadByte() v, bc := decodeFloat(src) ba := []byte{} switch { case math.IsNaN(v): return []byte("\"NaN\"") case math.IsInf(v, 1): return []byte("\"+Inf\"") case math.IsInf(v, -1): return []byte("\"-Inf\"") } if bc == isFloat32 { ba = strconv.AppendFloat(ba, v, 'f', -1, 32) } else if bc == isFloat64 { ba = strconv.AppendFloat(ba, v, 'f', -1, 64) } else { panic(fmt.Errorf("Invalid Float precision from decodeFloat: %d", bc)) } return ba default: panic(fmt.Errorf("Invalid Additional Type: %d in decodeSimpleFloat", minor)) } } func cbor2JsonOneObject(src *bufio.Reader, dst io.Writer) { pb, e := src.Peek(1) if e != nil { panic(e) } major := (pb[0] & maskOutAdditionalType) switch major { case majorTypeUnsignedInt: fallthrough case majorTypeNegativeInt: n := decodeInteger(src) dst.Write([]byte(strconv.Itoa(int(n)))) case majorTypeByteString: s := decodeString(src, false) dst.Write(s) case majorTypeUtf8String: s := decodeUTF8String(src) dst.Write(s) case majorTypeArray: array2Json(src, dst) case majorTypeMap: map2Json(src, dst) case majorTypeTags: s := decodeTagData(src) dst.Write(s) case majorTypeSimpleAndFloat: s := decodeSimpleFloat(src) dst.Write(s) } } func moreBytesToRead(src *bufio.Reader) bool { _, e := src.ReadByte() if e == nil { src.UnreadByte() return true } return false } // Cbor2JsonManyObjects decodes all the CBOR Objects read from src // reader. It keeps on decoding until reader returns EOF (error when reading). // Decoded string is written to the dst. At the end of every CBOR Object // newline is written to the output stream. // // Returns error (if any) that was encountered during decode. // The child functions will generate a panic when error is encountered and // this function will recover non-runtime Errors and return the reason as error. func Cbor2JsonManyObjects(src io.Reader, dst io.Writer) (err error) { defer func() { if r := recover(); r != nil { if _, ok := r.(runtime.Error); ok { panic(r) } err = r.(error) } }() bufRdr := bufio.NewReader(src) for moreBytesToRead(bufRdr) { cbor2JsonOneObject(bufRdr, dst) dst.Write([]byte("\n")) } return nil } // Detect if the bytes to be printed is Binary or not. func binaryFmt(p []byte) bool { if len(p) > 0 && p[0] > 0x7F { return true } return false } func getReader(str string) *bufio.Reader { return bufio.NewReader(strings.NewReader(str)) } // DecodeIfBinaryToString converts a binary formatted log msg to a // JSON formatted String Log message - suitable for printing to Console/Syslog. func DecodeIfBinaryToString(in []byte) string { if binaryFmt(in) { var b bytes.Buffer Cbor2JsonManyObjects(strings.NewReader(string(in)), &b) return b.String() } return string(in) } // DecodeObjectToStr checks if the input is a binary format, if so, // it will decode a single Object and return the decoded string. func DecodeObjectToStr(in []byte) string { if binaryFmt(in) { var b bytes.Buffer cbor2JsonOneObject(getReader(string(in)), &b) return b.String() } return string(in) } // DecodeIfBinaryToBytes checks if the input is a binary format, if so, // it will decode all Objects and return the decoded string as byte array. func DecodeIfBinaryToBytes(in []byte) []byte { if binaryFmt(in) { var b bytes.Buffer Cbor2JsonManyObjects(bytes.NewReader(in), &b) return b.Bytes() } return in }
package main import ( "compress/zlib" "flag" "io" "log" "os" "time" "github.com/rs/zerolog" ) func writeLog(fname string, count int, useCompress bool) { opFile := os.Stdout if fname != "<stdout>" { fil, _ := os.Create(fname) opFile = fil defer func() { if err := fil.Close(); err != nil { log.Fatal(err) } }() } var f io.WriteCloser = opFile if useCompress { f = zlib.NewWriter(f) defer func() { if err := f.Close(); err != nil { log.Fatal(err) } }() } zerolog.TimestampFunc = func() time.Time { return time.Now().Round(time.Second) } log := zerolog.New(f).With(). Timestamp(). Logger() for i := 0; i < count; i++ { log.Error(). Int("Fault", 41650+i).Msg("Some Message") } } func main() { outFile := flag.String("out", "<stdout>", "Output File to which logs will be written to (WILL overwrite if already present).") numLogs := flag.Int("num", 10, "Number of log messages to generate.") doCompress := flag.Bool("compress", false, "Enable inline compressed writer") flag.Parse() writeLog(*outFile, *numLogs, *doCompress) }
package cbor import "fmt" // AppendStrings encodes and adds an array of strings to the dst byte array. func (e Encoder) AppendStrings(dst []byte, vals []string) []byte { major := majorTypeArray l := len(vals) if l <= additionalMax { lb := byte(l) dst = append(dst, major|lb) } else { dst = appendCborTypePrefix(dst, major, uint64(l)) } for _, v := range vals { dst = e.AppendString(dst, v) } return dst } // AppendString encodes and adds a string to the dst byte array. func (Encoder) AppendString(dst []byte, s string) []byte { major := majorTypeUtf8String l := len(s) if l <= additionalMax { lb := byte(l) dst = append(dst, major|lb) } else { dst = appendCborTypePrefix(dst, majorTypeUtf8String, uint64(l)) } return append(dst, s...) } // AppendStringers encodes and adds an array of Stringer values // to the dst byte array. func (e Encoder) AppendStringers(dst []byte, vals []fmt.Stringer) []byte { if len(vals) == 0 { return e.AppendArrayEnd(e.AppendArrayStart(dst)) } dst = e.AppendArrayStart(dst) dst = e.AppendStringer(dst, vals[0]) if len(vals) > 1 { for _, val := range vals[1:] { dst = e.AppendStringer(dst, val) } } return e.AppendArrayEnd(dst) } // AppendStringer encodes and adds the Stringer value to the dst // byte array. func (e Encoder) AppendStringer(dst []byte, val fmt.Stringer) []byte { if val == nil { return e.AppendNil(dst) } return e.AppendString(dst, val.String()) } // AppendBytes encodes and adds an array of bytes to the dst byte array. func (Encoder) AppendBytes(dst, s []byte) []byte { major := majorTypeByteString l := len(s) if l <= additionalMax { lb := byte(l) dst = append(dst, major|lb) } else { dst = appendCborTypePrefix(dst, major, uint64(l)) } return append(dst, s...) } // AppendEmbeddedJSON adds a tag and embeds input JSON as such. func AppendEmbeddedJSON(dst, s []byte) []byte { major := majorTypeTags minor := additionalTypeEmbeddedJSON // Append the TAG to indicate this is Embedded JSON. dst = append(dst, major|additionalTypeIntUint16) dst = append(dst, byte(minor>>8)) dst = append(dst, byte(minor&0xff)) // Append the JSON Object as Byte String. major = majorTypeByteString l := len(s) if l <= additionalMax { lb := byte(l) dst = append(dst, major|lb) } else { dst = appendCborTypePrefix(dst, major, uint64(l)) } return append(dst, s...) } // AppendEmbeddedCBOR adds a tag and embeds input CBOR as such. func AppendEmbeddedCBOR(dst, s []byte) []byte { major := majorTypeTags minor := additionalTypeEmbeddedCBOR // Append the TAG to indicate this is Embedded JSON. dst = append(dst, major|additionalTypeIntUint8) dst = append(dst, minor) // Append the CBOR Object as Byte String. major = majorTypeByteString l := len(s) if l <= additionalMax { lb := byte(l) dst = append(dst, major|lb) } else { dst = appendCborTypePrefix(dst, major, uint64(l)) } return append(dst, s...) }
package cbor import ( "time" ) func appendIntegerTimestamp(dst []byte, t time.Time) []byte { major := majorTypeTags minor := additionalTypeTimestamp dst = append(dst, major|minor) secs := t.Unix() var val uint64 if secs < 0 { major = majorTypeNegativeInt val = uint64(-secs - 1) } else { major = majorTypeUnsignedInt val = uint64(secs) } dst = appendCborTypePrefix(dst, major, val) return dst } func (e Encoder) appendFloatTimestamp(dst []byte, t time.Time) []byte { major := majorTypeTags minor := additionalTypeTimestamp dst = append(dst, major|minor) secs := t.Unix() nanos := t.Nanosecond() var val float64 val = float64(secs)*1.0 + float64(nanos)*1e-9 return e.AppendFloat64(dst, val, -1) } // AppendTime encodes and adds a timestamp to the dst byte array. func (e Encoder) AppendTime(dst []byte, t time.Time, unused string) []byte { utc := t.UTC() if utc.Nanosecond() == 0 { return appendIntegerTimestamp(dst, utc) } return e.appendFloatTimestamp(dst, utc) } // AppendTimes encodes and adds an array of timestamps to the dst byte array. func (e Encoder) AppendTimes(dst []byte, vals []time.Time, unused string) []byte { major := majorTypeArray l := len(vals) if l == 0 { return e.AppendArrayEnd(e.AppendArrayStart(dst)) } if l <= additionalMax { lb := byte(l) dst = append(dst, major|lb) } else { dst = appendCborTypePrefix(dst, major, uint64(l)) } for _, t := range vals { dst = e.AppendTime(dst, t, unused) } return dst } // AppendDuration encodes and adds a duration to the dst byte array. // useInt field indicates whether to store the duration as seconds (integer) or // as seconds+nanoseconds (float). func (e Encoder) AppendDuration(dst []byte, d time.Duration, unit time.Duration, useInt bool, unused int) []byte { if useInt { return e.AppendInt64(dst, int64(d/unit)) } return e.AppendFloat64(dst, float64(d)/float64(unit), unused) } // AppendDurations encodes and adds an array of durations to the dst byte array. // useInt field indicates whether to store the duration as seconds (integer) or // as seconds+nanoseconds (float). func (e Encoder) AppendDurations(dst []byte, vals []time.Duration, unit time.Duration, useInt bool, unused int) []byte { major := majorTypeArray l := len(vals) if l == 0 { return e.AppendArrayEnd(e.AppendArrayStart(dst)) } if l <= additionalMax { lb := byte(l) dst = append(dst, major|lb) } else { dst = appendCborTypePrefix(dst, major, uint64(l)) } for _, d := range vals { dst = e.AppendDuration(dst, d, unit, useInt, unused) } return dst }
package cbor import ( "fmt" "math" "net" "reflect" ) // AppendNil inserts a 'Nil' object into the dst byte array. func (Encoder) AppendNil(dst []byte) []byte { return append(dst, majorTypeSimpleAndFloat|additionalTypeNull) } // AppendBeginMarker inserts a map start into the dst byte array. func (Encoder) AppendBeginMarker(dst []byte) []byte { return append(dst, majorTypeMap|additionalTypeInfiniteCount) } // AppendEndMarker inserts a map end into the dst byte array. func (Encoder) AppendEndMarker(dst []byte) []byte { return append(dst, majorTypeSimpleAndFloat|additionalTypeBreak) } // AppendObjectData takes an object in form of a byte array and appends to dst. func (Encoder) AppendObjectData(dst []byte, o []byte) []byte { // BeginMarker is present in the dst, which // should not be copied when appending to existing data. return append(dst, o[1:]...) } // AppendArrayStart adds markers to indicate the start of an array. func (Encoder) AppendArrayStart(dst []byte) []byte { return append(dst, majorTypeArray|additionalTypeInfiniteCount) } // AppendArrayEnd adds markers to indicate the end of an array. func (Encoder) AppendArrayEnd(dst []byte) []byte { return append(dst, majorTypeSimpleAndFloat|additionalTypeBreak) } // AppendArrayDelim adds markers to indicate end of a particular array element. func (Encoder) AppendArrayDelim(dst []byte) []byte { //No delimiters needed in cbor return dst } // AppendLineBreak is a noop that keep API compat with json encoder. func (Encoder) AppendLineBreak(dst []byte) []byte { // No line breaks needed in binary format. return dst } // AppendBool encodes and inserts a boolean value into the dst byte array. func (Encoder) AppendBool(dst []byte, val bool) []byte { b := additionalTypeBoolFalse if val { b = additionalTypeBoolTrue } return append(dst, majorTypeSimpleAndFloat|b) } // AppendBools encodes and inserts an array of boolean values into the dst byte array. func (e Encoder) AppendBools(dst []byte, vals []bool) []byte { major := majorTypeArray l := len(vals) if l == 0 { return e.AppendArrayEnd(e.AppendArrayStart(dst)) } if l <= additionalMax { lb := byte(l) dst = append(dst, major|lb) } else { dst = appendCborTypePrefix(dst, major, uint64(l)) } for _, v := range vals { dst = e.AppendBool(dst, v) } return dst } // AppendInt encodes and inserts an integer value into the dst byte array. func (Encoder) AppendInt(dst []byte, val int) []byte { major := majorTypeUnsignedInt contentVal := val if val < 0 { major = majorTypeNegativeInt contentVal = -val - 1 } if contentVal <= additionalMax { lb := byte(contentVal) dst = append(dst, major|lb) } else { dst = appendCborTypePrefix(dst, major, uint64(contentVal)) } return dst } // AppendInts encodes and inserts an array of integer values into the dst byte array. func (e Encoder) AppendInts(dst []byte, vals []int) []byte { major := majorTypeArray l := len(vals) if l == 0 { return e.AppendArrayEnd(e.AppendArrayStart(dst)) } if l <= additionalMax { lb := byte(l) dst = append(dst, major|lb) } else { dst = appendCborTypePrefix(dst, major, uint64(l)) } for _, v := range vals { dst = e.AppendInt(dst, v) } return dst } // AppendInt8 encodes and inserts an int8 value into the dst byte array. func (e Encoder) AppendInt8(dst []byte, val int8) []byte { return e.AppendInt(dst, int(val)) } // AppendInts8 encodes and inserts an array of integer values into the dst byte array. func (e Encoder) AppendInts8(dst []byte, vals []int8) []byte { major := majorTypeArray l := len(vals) if l == 0 { return e.AppendArrayEnd(e.AppendArrayStart(dst)) } if l <= additionalMax { lb := byte(l) dst = append(dst, major|lb) } else { dst = appendCborTypePrefix(dst, major, uint64(l)) } for _, v := range vals { dst = e.AppendInt(dst, int(v)) } return dst } // AppendInt16 encodes and inserts a int16 value into the dst byte array. func (e Encoder) AppendInt16(dst []byte, val int16) []byte { return e.AppendInt(dst, int(val)) } // AppendInts16 encodes and inserts an array of int16 values into the dst byte array. func (e Encoder) AppendInts16(dst []byte, vals []int16) []byte { major := majorTypeArray l := len(vals) if l == 0 { return e.AppendArrayEnd(e.AppendArrayStart(dst)) } if l <= additionalMax { lb := byte(l) dst = append(dst, major|lb) } else { dst = appendCborTypePrefix(dst, major, uint64(l)) } for _, v := range vals { dst = e.AppendInt(dst, int(v)) } return dst } // AppendInt32 encodes and inserts a int32 value into the dst byte array. func (e Encoder) AppendInt32(dst []byte, val int32) []byte { return e.AppendInt(dst, int(val)) } // AppendInts32 encodes and inserts an array of int32 values into the dst byte array. func (e Encoder) AppendInts32(dst []byte, vals []int32) []byte { major := majorTypeArray l := len(vals) if l == 0 { return e.AppendArrayEnd(e.AppendArrayStart(dst)) } if l <= additionalMax { lb := byte(l) dst = append(dst, major|lb) } else { dst = appendCborTypePrefix(dst, major, uint64(l)) } for _, v := range vals { dst = e.AppendInt(dst, int(v)) } return dst } // AppendInt64 encodes and inserts a int64 value into the dst byte array. func (Encoder) AppendInt64(dst []byte, val int64) []byte { major := majorTypeUnsignedInt contentVal := val if val < 0 { major = majorTypeNegativeInt contentVal = -val - 1 } if contentVal <= additionalMax { lb := byte(contentVal) dst = append(dst, major|lb) } else { dst = appendCborTypePrefix(dst, major, uint64(contentVal)) } return dst } // AppendInts64 encodes and inserts an array of int64 values into the dst byte array. func (e Encoder) AppendInts64(dst []byte, vals []int64) []byte { major := majorTypeArray l := len(vals) if l == 0 { return e.AppendArrayEnd(e.AppendArrayStart(dst)) } if l <= additionalMax { lb := byte(l) dst = append(dst, major|lb) } else { dst = appendCborTypePrefix(dst, major, uint64(l)) } for _, v := range vals { dst = e.AppendInt64(dst, v) } return dst } // AppendUint encodes and inserts an unsigned integer value into the dst byte array. func (e Encoder) AppendUint(dst []byte, val uint) []byte { return e.AppendInt64(dst, int64(val)) } // AppendUints encodes and inserts an array of unsigned integer values into the dst byte array. func (e Encoder) AppendUints(dst []byte, vals []uint) []byte { major := majorTypeArray l := len(vals) if l == 0 { return e.AppendArrayEnd(e.AppendArrayStart(dst)) } if l <= additionalMax { lb := byte(l) dst = append(dst, major|lb) } else { dst = appendCborTypePrefix(dst, major, uint64(l)) } for _, v := range vals { dst = e.AppendUint(dst, v) } return dst } // AppendUint8 encodes and inserts a unsigned int8 value into the dst byte array. func (e Encoder) AppendUint8(dst []byte, val uint8) []byte { return e.AppendUint(dst, uint(val)) } // AppendUints8 encodes and inserts an array of uint8 values into the dst byte array. func (e Encoder) AppendUints8(dst []byte, vals []uint8) []byte { major := majorTypeArray l := len(vals) if l == 0 { return e.AppendArrayEnd(e.AppendArrayStart(dst)) } if l <= additionalMax { lb := byte(l) dst = append(dst, major|lb) } else { dst = appendCborTypePrefix(dst, major, uint64(l)) } for _, v := range vals { dst = e.AppendUint8(dst, v) } return dst } // AppendUint16 encodes and inserts a uint16 value into the dst byte array. func (e Encoder) AppendUint16(dst []byte, val uint16) []byte { return e.AppendUint(dst, uint(val)) } // AppendUints16 encodes and inserts an array of uint16 values into the dst byte array. func (e Encoder) AppendUints16(dst []byte, vals []uint16) []byte { major := majorTypeArray l := len(vals) if l == 0 { return e.AppendArrayEnd(e.AppendArrayStart(dst)) } if l <= additionalMax { lb := byte(l) dst = append(dst, major|lb) } else { dst = appendCborTypePrefix(dst, major, uint64(l)) } for _, v := range vals { dst = e.AppendUint16(dst, v) } return dst } // AppendUint32 encodes and inserts a uint32 value into the dst byte array. func (e Encoder) AppendUint32(dst []byte, val uint32) []byte { return e.AppendUint(dst, uint(val)) } // AppendUints32 encodes and inserts an array of uint32 values into the dst byte array. func (e Encoder) AppendUints32(dst []byte, vals []uint32) []byte { major := majorTypeArray l := len(vals) if l == 0 { return e.AppendArrayEnd(e.AppendArrayStart(dst)) } if l <= additionalMax { lb := byte(l) dst = append(dst, major|lb) } else { dst = appendCborTypePrefix(dst, major, uint64(l)) } for _, v := range vals { dst = e.AppendUint32(dst, v) } return dst } // AppendUint64 encodes and inserts a uint64 value into the dst byte array. func (Encoder) AppendUint64(dst []byte, val uint64) []byte { major := majorTypeUnsignedInt contentVal := val if contentVal <= additionalMax { lb := byte(contentVal) dst = append(dst, major|lb) } else { dst = appendCborTypePrefix(dst, major, contentVal) } return dst } // AppendUints64 encodes and inserts an array of uint64 values into the dst byte array. func (e Encoder) AppendUints64(dst []byte, vals []uint64) []byte { major := majorTypeArray l := len(vals) if l == 0 { return e.AppendArrayEnd(e.AppendArrayStart(dst)) } if l <= additionalMax { lb := byte(l) dst = append(dst, major|lb) } else { dst = appendCborTypePrefix(dst, major, uint64(l)) } for _, v := range vals { dst = e.AppendUint64(dst, v) } return dst } // AppendFloat32 encodes and inserts a single precision float value into the dst byte array. func (Encoder) AppendFloat32(dst []byte, val float32, unused int) []byte { switch { case math.IsNaN(float64(val)): return append(dst, "\xfa\x7f\xc0\x00\x00"...) case math.IsInf(float64(val), 1): return append(dst, "\xfa\x7f\x80\x00\x00"...) case math.IsInf(float64(val), -1): return append(dst, "\xfa\xff\x80\x00\x00"...) } major := majorTypeSimpleAndFloat subType := additionalTypeFloat32 n := math.Float32bits(val) var buf [4]byte for i := uint(0); i < 4; i++ { buf[i] = byte(n >> ((3 - i) * 8)) } return append(append(dst, major|subType), buf[0], buf[1], buf[2], buf[3]) } // AppendFloats32 encodes and inserts an array of single precision float value into the dst byte array. func (e Encoder) AppendFloats32(dst []byte, vals []float32, unused int) []byte { major := majorTypeArray l := len(vals) if l == 0 { return e.AppendArrayEnd(e.AppendArrayStart(dst)) } if l <= additionalMax { lb := byte(l) dst = append(dst, major|lb) } else { dst = appendCborTypePrefix(dst, major, uint64(l)) } for _, v := range vals { dst = e.AppendFloat32(dst, v, unused) } return dst } // AppendFloat64 encodes and inserts a double precision float value into the dst byte array. func (Encoder) AppendFloat64(dst []byte, val float64, unused int) []byte { switch { case math.IsNaN(val): return append(dst, "\xfb\x7f\xf8\x00\x00\x00\x00\x00\x00"...) case math.IsInf(val, 1): return append(dst, "\xfb\x7f\xf0\x00\x00\x00\x00\x00\x00"...) case math.IsInf(val, -1): return append(dst, "\xfb\xff\xf0\x00\x00\x00\x00\x00\x00"...) } major := majorTypeSimpleAndFloat subType := additionalTypeFloat64 n := math.Float64bits(val) dst = append(dst, major|subType) for i := uint(1); i <= 8; i++ { b := byte(n >> ((8 - i) * 8)) dst = append(dst, b) } return dst } // AppendFloats64 encodes and inserts an array of double precision float values into the dst byte array. func (e Encoder) AppendFloats64(dst []byte, vals []float64, unused int) []byte { major := majorTypeArray l := len(vals) if l == 0 { return e.AppendArrayEnd(e.AppendArrayStart(dst)) } if l <= additionalMax { lb := byte(l) dst = append(dst, major|lb) } else { dst = appendCborTypePrefix(dst, major, uint64(l)) } for _, v := range vals { dst = e.AppendFloat64(dst, v, unused) } return dst } // AppendInterface takes an arbitrary object and converts it to JSON and embeds it dst. func (e Encoder) AppendInterface(dst []byte, i interface{}) []byte { marshaled, err := JSONMarshalFunc(i) if err != nil { return e.AppendString(dst, fmt.Sprintf("marshaling error: %v", err)) } return AppendEmbeddedJSON(dst, marshaled) } // AppendType appends the parameter type (as a string) to the input byte slice. func (e Encoder) AppendType(dst []byte, i interface{}) []byte { if i == nil { return e.AppendString(dst, "<nil>") } return e.AppendString(dst, reflect.TypeOf(i).String()) } // AppendIPAddr encodes and inserts an IP Address (IPv4 or IPv6). func (e Encoder) AppendIPAddr(dst []byte, ip net.IP) []byte { dst = append(dst, majorTypeTags|additionalTypeIntUint16) dst = append(dst, byte(additionalTypeTagNetworkAddr>>8)) dst = append(dst, byte(additionalTypeTagNetworkAddr&0xff)) return e.AppendBytes(dst, ip) } // AppendIPPrefix encodes and inserts an IP Address Prefix (Address + Mask Length). func (e Encoder) AppendIPPrefix(dst []byte, pfx net.IPNet) []byte { dst = append(dst, majorTypeTags|additionalTypeIntUint16) dst = append(dst, byte(additionalTypeTagNetworkPrefix>>8)) dst = append(dst, byte(additionalTypeTagNetworkPrefix&0xff)) // Prefix is a tuple (aka MAP of 1 pair of elements) - // first element is prefix, second is mask length. dst = append(dst, majorTypeMap|0x1) dst = e.AppendBytes(dst, pfx.IP) maskLen, _ := pfx.Mask.Size() return e.AppendUint8(dst, uint8(maskLen)) } // AppendMACAddr encodes and inserts a Hardware (MAC) address. func (e Encoder) AppendMACAddr(dst []byte, ha net.HardwareAddr) []byte { dst = append(dst, majorTypeTags|additionalTypeIntUint16) dst = append(dst, byte(additionalTypeTagNetworkAddr>>8)) dst = append(dst, byte(additionalTypeTagNetworkAddr&0xff)) return e.AppendBytes(dst, ha) } // AppendHex adds a TAG and inserts a hex bytes as a string. func (e Encoder) AppendHex(dst []byte, val []byte) []byte { dst = append(dst, majorTypeTags|additionalTypeIntUint16) dst = append(dst, byte(additionalTypeTagHexString>>8)) dst = append(dst, byte(additionalTypeTagHexString&0xff)) return e.AppendBytes(dst, val) }
package json // JSONMarshalFunc is used to marshal interface to JSON encoded byte slice. // Making it package level instead of embedded in Encoder brings // some extra efforts at importing, but avoids value copy when the functions // of Encoder being invoked. // DO REMEMBER to set this variable at importing, or // you might get a nil pointer dereference panic at runtime. var JSONMarshalFunc func(v interface{}) ([]byte, error) type Encoder struct{} // AppendKey appends a new key to the output JSON. func (e Encoder) AppendKey(dst []byte, key string) []byte { if dst[len(dst)-1] != '{' { dst = append(dst, ',') } return append(e.AppendString(dst, key), ':') }
package json import "unicode/utf8" // AppendBytes is a mirror of appendString with []byte arg func (Encoder) AppendBytes(dst, s []byte) []byte { dst = append(dst, '"') for i := 0; i < len(s); i++ { if !noEscapeTable[s[i]] { dst = appendBytesComplex(dst, s, i) return append(dst, '"') } } dst = append(dst, s...) return append(dst, '"') } // AppendHex encodes the input bytes to a hex string and appends // the encoded string to the input byte slice. // // The operation loops though each byte and encodes it as hex using // the hex lookup table. func (Encoder) AppendHex(dst, s []byte) []byte { dst = append(dst, '"') for _, v := range s { dst = append(dst, hex[v>>4], hex[v&0x0f]) } return append(dst, '"') } // appendBytesComplex is a mirror of the appendStringComplex // with []byte arg func appendBytesComplex(dst, s []byte, i int) []byte { start := 0 for i < len(s) { b := s[i] if b >= utf8.RuneSelf { r, size := utf8.DecodeRune(s[i:]) if r == utf8.RuneError && size == 1 { if start < i { dst = append(dst, s[start:i]...) } dst = append(dst, `\ufffd`...) i += size start = i continue } i += size continue } if noEscapeTable[b] { i++ continue } // We encountered a character that needs to be encoded. // Let's append the previous simple characters to the byte slice // and switch our operation to read and encode the remainder // characters byte-by-byte. if start < i { dst = append(dst, s[start:i]...) } switch b { case '"', '\\': dst = append(dst, '\\', b) case '\b': dst = append(dst, '\\', 'b') case '\f': dst = append(dst, '\\', 'f') case '\n': dst = append(dst, '\\', 'n') case '\r': dst = append(dst, '\\', 'r') case '\t': dst = append(dst, '\\', 't') default: dst = append(dst, '\\', 'u', '0', '0', hex[b>>4], hex[b&0xF]) } i++ start = i } if start < len(s) { dst = append(dst, s[start:]...) } return dst }
package json import ( "fmt" "unicode/utf8" ) const hex = "0123456789abcdef" var noEscapeTable = [256]bool{} func init() { for i := 0; i <= 0x7e; i++ { noEscapeTable[i] = i >= 0x20 && i != '\\' && i != '"' } } // AppendStrings encodes the input strings to json and // appends the encoded string list to the input byte slice. func (e Encoder) AppendStrings(dst []byte, vals []string) []byte { if len(vals) == 0 { return append(dst, '[', ']') } dst = append(dst, '[') dst = e.AppendString(dst, vals[0]) if len(vals) > 1 { for _, val := range vals[1:] { dst = e.AppendString(append(dst, ','), val) } } dst = append(dst, ']') return dst } // AppendString encodes the input string to json and appends // the encoded string to the input byte slice. // // The operation loops though each byte in the string looking // for characters that need json or utf8 encoding. If the string // does not need encoding, then the string is appended in its // entirety to the byte slice. // If we encounter a byte that does need encoding, switch up // the operation and perform a byte-by-byte read-encode-append. func (Encoder) AppendString(dst []byte, s string) []byte { // Start with a double quote. dst = append(dst, '"') // Loop through each character in the string. for i := 0; i < len(s); i++ { // Check if the character needs encoding. Control characters, slashes, // and the double quote need json encoding. Bytes above the ascii // boundary needs utf8 encoding. if !noEscapeTable[s[i]] { // We encountered a character that needs to be encoded. Switch // to complex version of the algorithm. dst = appendStringComplex(dst, s, i) return append(dst, '"') } } // The string has no need for encoding and therefore is directly // appended to the byte slice. dst = append(dst, s...) // End with a double quote return append(dst, '"') } // AppendStringers encodes the provided Stringer list to json and // appends the encoded Stringer list to the input byte slice. func (e Encoder) AppendStringers(dst []byte, vals []fmt.Stringer) []byte { if len(vals) == 0 { return append(dst, '[', ']') } dst = append(dst, '[') dst = e.AppendStringer(dst, vals[0]) if len(vals) > 1 { for _, val := range vals[1:] { dst = e.AppendStringer(append(dst, ','), val) } } return append(dst, ']') } // AppendStringer encodes the input Stringer to json and appends the // encoded Stringer value to the input byte slice. func (e Encoder) AppendStringer(dst []byte, val fmt.Stringer) []byte { if val == nil { return e.AppendInterface(dst, nil) } return e.AppendString(dst, val.String()) } //// appendStringComplex is used by appendString to take over an in // progress JSON string encoding that encountered a character that needs // to be encoded. func appendStringComplex(dst []byte, s string, i int) []byte { start := 0 for i < len(s) { b := s[i] if b >= utf8.RuneSelf { r, size := utf8.DecodeRuneInString(s[i:]) if r == utf8.RuneError && size == 1 { // In case of error, first append previous simple characters to // the byte slice if any and append a replacement character code // in place of the invalid sequence. if start < i { dst = append(dst, s[start:i]...) } dst = append(dst, `\ufffd`...) i += size start = i continue } i += size continue } if noEscapeTable[b] { i++ continue } // We encountered a character that needs to be encoded. // Let's append the previous simple characters to the byte slice // and switch our operation to read and encode the remainder // characters byte-by-byte. if start < i { dst = append(dst, s[start:i]...) } switch b { case '"', '\\': dst = append(dst, '\\', b) case '\b': dst = append(dst, '\\', 'b') case '\f': dst = append(dst, '\\', 'f') case '\n': dst = append(dst, '\\', 'n') case '\r': dst = append(dst, '\\', 'r') case '\t': dst = append(dst, '\\', 't') default: dst = append(dst, '\\', 'u', '0', '0', hex[b>>4], hex[b&0xF]) } i++ start = i } if start < len(s) { dst = append(dst, s[start:]...) } return dst }
package json import ( "strconv" "time" ) const ( // Import from zerolog/global.go timeFormatUnix = "" timeFormatUnixMs = "UNIXMS" timeFormatUnixMicro = "UNIXMICRO" timeFormatUnixNano = "UNIXNANO" ) // AppendTime formats the input time with the given format // and appends the encoded string to the input byte slice. func (e Encoder) AppendTime(dst []byte, t time.Time, format string) []byte { switch format { case timeFormatUnix: return e.AppendInt64(dst, t.Unix()) case timeFormatUnixMs: return e.AppendInt64(dst, t.UnixNano()/1000000) case timeFormatUnixMicro: return e.AppendInt64(dst, t.UnixNano()/1000) case timeFormatUnixNano: return e.AppendInt64(dst, t.UnixNano()) } return append(t.AppendFormat(append(dst, '"'), format), '"') } // AppendTimes converts the input times with the given format // and appends the encoded string list to the input byte slice. func (Encoder) AppendTimes(dst []byte, vals []time.Time, format string) []byte { switch format { case timeFormatUnix: return appendUnixTimes(dst, vals) case timeFormatUnixMs: return appendUnixNanoTimes(dst, vals, 1000000) case timeFormatUnixMicro: return appendUnixNanoTimes(dst, vals, 1000) case timeFormatUnixNano: return appendUnixNanoTimes(dst, vals, 1) } if len(vals) == 0 { return append(dst, '[', ']') } dst = append(dst, '[') dst = append(vals[0].AppendFormat(append(dst, '"'), format), '"') if len(vals) > 1 { for _, t := range vals[1:] { dst = append(t.AppendFormat(append(dst, ',', '"'), format), '"') } } dst = append(dst, ']') return dst } func appendUnixTimes(dst []byte, vals []time.Time) []byte { if len(vals) == 0 { return append(dst, '[', ']') } dst = append(dst, '[') dst = strconv.AppendInt(dst, vals[0].Unix(), 10) if len(vals) > 1 { for _, t := range vals[1:] { dst = strconv.AppendInt(append(dst, ','), t.Unix(), 10) } } dst = append(dst, ']') return dst } func appendUnixNanoTimes(dst []byte, vals []time.Time, div int64) []byte { if len(vals) == 0 { return append(dst, '[', ']') } dst = append(dst, '[') dst = strconv.AppendInt(dst, vals[0].UnixNano()/div, 10) if len(vals) > 1 { for _, t := range vals[1:] { dst = strconv.AppendInt(append(dst, ','), t.UnixNano()/div, 10) } } dst = append(dst, ']') return dst } // AppendDuration formats the input duration with the given unit & format // and appends the encoded string to the input byte slice. func (e Encoder) AppendDuration(dst []byte, d time.Duration, unit time.Duration, useInt bool, precision int) []byte { if useInt { return strconv.AppendInt(dst, int64(d/unit), 10) } return e.AppendFloat64(dst, float64(d)/float64(unit), precision) } // AppendDurations formats the input durations with the given unit & format // and appends the encoded string list to the input byte slice. func (e Encoder) AppendDurations(dst []byte, vals []time.Duration, unit time.Duration, useInt bool, precision int) []byte { if len(vals) == 0 { return append(dst, '[', ']') } dst = append(dst, '[') dst = e.AppendDuration(dst, vals[0], unit, useInt, precision) if len(vals) > 1 { for _, d := range vals[1:] { dst = e.AppendDuration(append(dst, ','), d, unit, useInt, precision) } } dst = append(dst, ']') return dst }
package json import ( "fmt" "math" "net" "reflect" "strconv" ) // AppendNil inserts a 'Nil' object into the dst byte array. func (Encoder) AppendNil(dst []byte) []byte { return append(dst, "null"...) } // AppendBeginMarker inserts a map start into the dst byte array. func (Encoder) AppendBeginMarker(dst []byte) []byte { return append(dst, '{') } // AppendEndMarker inserts a map end into the dst byte array. func (Encoder) AppendEndMarker(dst []byte) []byte { return append(dst, '}') } // AppendLineBreak appends a line break. func (Encoder) AppendLineBreak(dst []byte) []byte { return append(dst, '\n') } // AppendArrayStart adds markers to indicate the start of an array. func (Encoder) AppendArrayStart(dst []byte) []byte { return append(dst, '[') } // AppendArrayEnd adds markers to indicate the end of an array. func (Encoder) AppendArrayEnd(dst []byte) []byte { return append(dst, ']') } // AppendArrayDelim adds markers to indicate end of a particular array element. func (Encoder) AppendArrayDelim(dst []byte) []byte { if len(dst) > 0 { return append(dst, ',') } return dst } // AppendBool converts the input bool to a string and // appends the encoded string to the input byte slice. func (Encoder) AppendBool(dst []byte, val bool) []byte { return strconv.AppendBool(dst, val) } // AppendBools encodes the input bools to json and // appends the encoded string list to the input byte slice. func (Encoder) AppendBools(dst []byte, vals []bool) []byte { if len(vals) == 0 { return append(dst, '[', ']') } dst = append(dst, '[') dst = strconv.AppendBool(dst, vals[0]) if len(vals) > 1 { for _, val := range vals[1:] { dst = strconv.AppendBool(append(dst, ','), val) } } dst = append(dst, ']') return dst } // AppendInt converts the input int to a string and // appends the encoded string to the input byte slice. func (Encoder) AppendInt(dst []byte, val int) []byte { return strconv.AppendInt(dst, int64(val), 10) } // AppendInts encodes the input ints to json and // appends the encoded string list to the input byte slice. func (Encoder) AppendInts(dst []byte, vals []int) []byte { if len(vals) == 0 { return append(dst, '[', ']') } dst = append(dst, '[') dst = strconv.AppendInt(dst, int64(vals[0]), 10) if len(vals) > 1 { for _, val := range vals[1:] { dst = strconv.AppendInt(append(dst, ','), int64(val), 10) } } dst = append(dst, ']') return dst } // AppendInt8 converts the input []int8 to a string and // appends the encoded string to the input byte slice. func (Encoder) AppendInt8(dst []byte, val int8) []byte { return strconv.AppendInt(dst, int64(val), 10) } // AppendInts8 encodes the input int8s to json and // appends the encoded string list to the input byte slice. func (Encoder) AppendInts8(dst []byte, vals []int8) []byte { if len(vals) == 0 { return append(dst, '[', ']') } dst = append(dst, '[') dst = strconv.AppendInt(dst, int64(vals[0]), 10) if len(vals) > 1 { for _, val := range vals[1:] { dst = strconv.AppendInt(append(dst, ','), int64(val), 10) } } dst = append(dst, ']') return dst } // AppendInt16 converts the input int16 to a string and // appends the encoded string to the input byte slice. func (Encoder) AppendInt16(dst []byte, val int16) []byte { return strconv.AppendInt(dst, int64(val), 10) } // AppendInts16 encodes the input int16s to json and // appends the encoded string list to the input byte slice. func (Encoder) AppendInts16(dst []byte, vals []int16) []byte { if len(vals) == 0 { return append(dst, '[', ']') } dst = append(dst, '[') dst = strconv.AppendInt(dst, int64(vals[0]), 10) if len(vals) > 1 { for _, val := range vals[1:] { dst = strconv.AppendInt(append(dst, ','), int64(val), 10) } } dst = append(dst, ']') return dst } // AppendInt32 converts the input int32 to a string and // appends the encoded string to the input byte slice. func (Encoder) AppendInt32(dst []byte, val int32) []byte { return strconv.AppendInt(dst, int64(val), 10) } // AppendInts32 encodes the input int32s to json and // appends the encoded string list to the input byte slice. func (Encoder) AppendInts32(dst []byte, vals []int32) []byte { if len(vals) == 0 { return append(dst, '[', ']') } dst = append(dst, '[') dst = strconv.AppendInt(dst, int64(vals[0]), 10) if len(vals) > 1 { for _, val := range vals[1:] { dst = strconv.AppendInt(append(dst, ','), int64(val), 10) } } dst = append(dst, ']') return dst } // AppendInt64 converts the input int64 to a string and // appends the encoded string to the input byte slice. func (Encoder) AppendInt64(dst []byte, val int64) []byte { return strconv.AppendInt(dst, val, 10) } // AppendInts64 encodes the input int64s to json and // appends the encoded string list to the input byte slice. func (Encoder) AppendInts64(dst []byte, vals []int64) []byte { if len(vals) == 0 { return append(dst, '[', ']') } dst = append(dst, '[') dst = strconv.AppendInt(dst, vals[0], 10) if len(vals) > 1 { for _, val := range vals[1:] { dst = strconv.AppendInt(append(dst, ','), val, 10) } } dst = append(dst, ']') return dst } // AppendUint converts the input uint to a string and // appends the encoded string to the input byte slice. func (Encoder) AppendUint(dst []byte, val uint) []byte { return strconv.AppendUint(dst, uint64(val), 10) } // AppendUints encodes the input uints to json and // appends the encoded string list to the input byte slice. func (Encoder) AppendUints(dst []byte, vals []uint) []byte { if len(vals) == 0 { return append(dst, '[', ']') } dst = append(dst, '[') dst = strconv.AppendUint(dst, uint64(vals[0]), 10) if len(vals) > 1 { for _, val := range vals[1:] { dst = strconv.AppendUint(append(dst, ','), uint64(val), 10) } } dst = append(dst, ']') return dst } // AppendUint8 converts the input uint8 to a string and // appends the encoded string to the input byte slice. func (Encoder) AppendUint8(dst []byte, val uint8) []byte { return strconv.AppendUint(dst, uint64(val), 10) } // AppendUints8 encodes the input uint8s to json and // appends the encoded string list to the input byte slice. func (Encoder) AppendUints8(dst []byte, vals []uint8) []byte { if len(vals) == 0 { return append(dst, '[', ']') } dst = append(dst, '[') dst = strconv.AppendUint(dst, uint64(vals[0]), 10) if len(vals) > 1 { for _, val := range vals[1:] { dst = strconv.AppendUint(append(dst, ','), uint64(val), 10) } } dst = append(dst, ']') return dst } // AppendUint16 converts the input uint16 to a string and // appends the encoded string to the input byte slice. func (Encoder) AppendUint16(dst []byte, val uint16) []byte { return strconv.AppendUint(dst, uint64(val), 10) } // AppendUints16 encodes the input uint16s to json and // appends the encoded string list to the input byte slice. func (Encoder) AppendUints16(dst []byte, vals []uint16) []byte { if len(vals) == 0 { return append(dst, '[', ']') } dst = append(dst, '[') dst = strconv.AppendUint(dst, uint64(vals[0]), 10) if len(vals) > 1 { for _, val := range vals[1:] { dst = strconv.AppendUint(append(dst, ','), uint64(val), 10) } } dst = append(dst, ']') return dst } // AppendUint32 converts the input uint32 to a string and // appends the encoded string to the input byte slice. func (Encoder) AppendUint32(dst []byte, val uint32) []byte { return strconv.AppendUint(dst, uint64(val), 10) } // AppendUints32 encodes the input uint32s to json and // appends the encoded string list to the input byte slice. func (Encoder) AppendUints32(dst []byte, vals []uint32) []byte { if len(vals) == 0 { return append(dst, '[', ']') } dst = append(dst, '[') dst = strconv.AppendUint(dst, uint64(vals[0]), 10) if len(vals) > 1 { for _, val := range vals[1:] { dst = strconv.AppendUint(append(dst, ','), uint64(val), 10) } } dst = append(dst, ']') return dst } // AppendUint64 converts the input uint64 to a string and // appends the encoded string to the input byte slice. func (Encoder) AppendUint64(dst []byte, val uint64) []byte { return strconv.AppendUint(dst, val, 10) } // AppendUints64 encodes the input uint64s to json and // appends the encoded string list to the input byte slice. func (Encoder) AppendUints64(dst []byte, vals []uint64) []byte { if len(vals) == 0 { return append(dst, '[', ']') } dst = append(dst, '[') dst = strconv.AppendUint(dst, vals[0], 10) if len(vals) > 1 { for _, val := range vals[1:] { dst = strconv.AppendUint(append(dst, ','), val, 10) } } dst = append(dst, ']') return dst } func appendFloat(dst []byte, val float64, bitSize, precision int) []byte { // JSON does not permit NaN or Infinity. A typical JSON encoder would fail // with an error, but a logging library wants the data to get through so we // make a tradeoff and store those types as string. switch { case math.IsNaN(val): return append(dst, `"NaN"`...) case math.IsInf(val, 1): return append(dst, `"+Inf"`...) case math.IsInf(val, -1): return append(dst, `"-Inf"`...) } // convert as if by es6 number to string conversion // see also https://cs.opensource.google/go/go/+/refs/tags/go1.20.3:src/encoding/json/encode.go;l=573 strFmt := byte('f') // If precision is set to a value other than -1, we always just format the float using that precision. if precision == -1 { // Use float32 comparisons for underlying float32 value to get precise cutoffs right. if abs := math.Abs(val); abs != 0 { if bitSize == 64 && (abs < 1e-6 || abs >= 1e21) || bitSize == 32 && (float32(abs) < 1e-6 || float32(abs) >= 1e21) { strFmt = 'e' } } } dst = strconv.AppendFloat(dst, val, strFmt, precision, bitSize) if strFmt == 'e' { // Clean up e-09 to e-9 n := len(dst) if n >= 4 && dst[n-4] == 'e' && dst[n-3] == '-' && dst[n-2] == '0' { dst[n-2] = dst[n-1] dst = dst[:n-1] } } return dst } // AppendFloat32 converts the input float32 to a string and // appends the encoded string to the input byte slice. func (Encoder) AppendFloat32(dst []byte, val float32, precision int) []byte { return appendFloat(dst, float64(val), 32, precision) } // AppendFloats32 encodes the input float32s to json and // appends the encoded string list to the input byte slice. func (Encoder) AppendFloats32(dst []byte, vals []float32, precision int) []byte { if len(vals) == 0 { return append(dst, '[', ']') } dst = append(dst, '[') dst = appendFloat(dst, float64(vals[0]), 32, precision) if len(vals) > 1 { for _, val := range vals[1:] { dst = appendFloat(append(dst, ','), float64(val), 32, precision) } } dst = append(dst, ']') return dst } // AppendFloat64 converts the input float64 to a string and // appends the encoded string to the input byte slice. func (Encoder) AppendFloat64(dst []byte, val float64, precision int) []byte { return appendFloat(dst, val, 64, precision) } // AppendFloats64 encodes the input float64s to json and // appends the encoded string list to the input byte slice. func (Encoder) AppendFloats64(dst []byte, vals []float64, precision int) []byte { if len(vals) == 0 { return append(dst, '[', ']') } dst = append(dst, '[') dst = appendFloat(dst, vals[0], 64, precision) if len(vals) > 1 { for _, val := range vals[1:] { dst = appendFloat(append(dst, ','), val, 64, precision) } } dst = append(dst, ']') return dst } // AppendInterface marshals the input interface to a string and // appends the encoded string to the input byte slice. func (e Encoder) AppendInterface(dst []byte, i interface{}) []byte { marshaled, err := JSONMarshalFunc(i) if err != nil { return e.AppendString(dst, fmt.Sprintf("marshaling error: %v", err)) } return append(dst, marshaled...) } // AppendType appends the parameter type (as a string) to the input byte slice. func (e Encoder) AppendType(dst []byte, i interface{}) []byte { if i == nil { return e.AppendString(dst, "<nil>") } return e.AppendString(dst, reflect.TypeOf(i).String()) } // AppendObjectData takes in an object that is already in a byte array // and adds it to the dst. func (Encoder) AppendObjectData(dst []byte, o []byte) []byte { // Three conditions apply here: // 1. new content starts with '{' - which should be dropped OR // 2. new content starts with '{' - which should be replaced with ',' // to separate with existing content OR // 3. existing content has already other fields if o[0] == '{' { if len(dst) > 1 { dst = append(dst, ',') } o = o[1:] } else if len(dst) > 1 { dst = append(dst, ',') } return append(dst, o...) } // AppendIPAddr adds IPv4 or IPv6 address to dst. func (e Encoder) AppendIPAddr(dst []byte, ip net.IP) []byte { return e.AppendString(dst, ip.String()) } // AppendIPPrefix adds IPv4 or IPv6 Prefix (address & mask) to dst. func (e Encoder) AppendIPPrefix(dst []byte, pfx net.IPNet) []byte { return e.AppendString(dst, pfx.String()) } // AppendMACAddr adds MAC address to dst. func (e Encoder) AppendMACAddr(dst []byte, ha net.HardwareAddr) []byte { return e.AppendString(dst, ha.String()) }
//go:build !windows // +build !windows // Package journald provides a io.Writer to send the logs // to journalD component of systemd. package journald // This file provides a zerolog writer so that logs printed // using zerolog library can be sent to a journalD. // Zerolog's Top level key/Value Pairs are translated to // journald's args - all Values are sent to journald as strings. // And all key strings are converted to uppercase before sending // to journald (as required by journald). // In addition, entire log message (all Key Value Pairs), is also // sent to journald under the key "JSON". import ( "bytes" "encoding/json" "fmt" "io" "strings" "github.com/coreos/go-systemd/v22/journal" "github.com/rs/zerolog" "github.com/rs/zerolog/internal/cbor" ) const defaultJournalDPrio = journal.PriNotice // NewJournalDWriter returns a zerolog log destination // to be used as parameter to New() calls. Writing logs // to this writer will send the log messages to journalD // running in this system. func NewJournalDWriter() io.Writer { return journalWriter{} } type journalWriter struct { } // levelToJPrio converts zerolog Level string into // journalD's priority values. JournalD has more // priorities than zerolog. func levelToJPrio(zLevel string) journal.Priority { lvl, _ := zerolog.ParseLevel(zLevel) switch lvl { case zerolog.TraceLevel: return journal.PriDebug case zerolog.DebugLevel: return journal.PriDebug case zerolog.InfoLevel: return journal.PriInfo case zerolog.WarnLevel: return journal.PriWarning case zerolog.ErrorLevel: return journal.PriErr case zerolog.FatalLevel: return journal.PriCrit case zerolog.PanicLevel: return journal.PriEmerg case zerolog.NoLevel: return journal.PriNotice } return defaultJournalDPrio } func (w journalWriter) Write(p []byte) (n int, err error) { var event map[string]interface{} origPLen := len(p) p = cbor.DecodeIfBinaryToBytes(p) d := json.NewDecoder(bytes.NewReader(p)) d.UseNumber() err = d.Decode(&event) jPrio := defaultJournalDPrio args := make(map[string]string) if err != nil { return } if l, ok := event[zerolog.LevelFieldName].(string); ok { jPrio = levelToJPrio(l) } msg := "" for key, value := range event { jKey := strings.ToUpper(key) switch key { case zerolog.LevelFieldName, zerolog.TimestampFieldName: continue case zerolog.MessageFieldName: msg, _ = value.(string) continue } switch v := value.(type) { case string: args[jKey] = v case json.Number: args[jKey] = fmt.Sprint(value) default: b, err := zerolog.InterfaceMarshalFunc(value) if err != nil { args[jKey] = fmt.Sprintf("[error: %v]", err) } else { args[jKey] = string(b) } } } args["JSON"] = string(p) err = journal.Send(msg, jPrio, args) if err == nil { n = origPLen } return }
// Package zerolog provides a lightweight logging library dedicated to JSON logging. // // A global Logger can be use for simple logging: // // import "github.com/rs/zerolog/log" // // log.Info().Msg("hello world") // // Output: {"time":1494567715,"level":"info","message":"hello world"} // // NOTE: To import the global logger, import the "log" subpackage "github.com/rs/zerolog/log". // // Fields can be added to log messages: // // log.Info().Str("foo", "bar").Msg("hello world") // // Output: {"time":1494567715,"level":"info","message":"hello world","foo":"bar"} // // Create logger instance to manage different outputs: // // logger := zerolog.New(os.Stderr).With().Timestamp().Logger() // logger.Info(). // Str("foo", "bar"). // Msg("hello world") // // Output: {"time":1494567715,"level":"info","message":"hello world","foo":"bar"} // // Sub-loggers let you chain loggers with additional context: // // sublogger := log.With().Str("component", "foo").Logger() // sublogger.Info().Msg("hello world") // // Output: {"time":1494567715,"level":"info","message":"hello world","component":"foo"} // // Level logging // // zerolog.SetGlobalLevel(zerolog.InfoLevel) // // log.Debug().Msg("filtered out message") // log.Info().Msg("routed message") // // if e := log.Debug(); e.Enabled() { // // Compute log output only if enabled. // value := compute() // e.Str("foo": value).Msg("some debug message") // } // // Output: {"level":"info","time":1494567715,"routed message"} // // Customize automatic field names: // // log.TimestampFieldName = "t" // log.LevelFieldName = "p" // log.MessageFieldName = "m" // // log.Info().Msg("hello world") // // Output: {"t":1494567715,"p":"info","m":"hello world"} // // Log with no level and message: // // log.Log().Str("foo","bar").Msg("") // // Output: {"time":1494567715,"foo":"bar"} // // Add contextual fields to global Logger: // // log.Logger = log.With().Str("foo", "bar").Logger() // // Sample logs: // // sampled := log.Sample(&zerolog.BasicSampler{N: 10}) // sampled.Info().Msg("will be logged every 10 messages") // // Log with contextual hooks: // // // Create the hook: // type SeverityHook struct{} // // func (h SeverityHook) Run(e *zerolog.Event, level zerolog.Level, msg string) { // if level != zerolog.NoLevel { // e.Str("severity", level.String()) // } // } // // // And use it: // var h SeverityHook // log := zerolog.New(os.Stdout).Hook(h) // log.Warn().Msg("") // // Output: {"level":"warn","severity":"warn"} // // # Caveats // // Field duplication: // // There is no fields deduplication out-of-the-box. // Using the same key multiple times creates new key in final JSON each time. // // logger := zerolog.New(os.Stderr).With().Timestamp().Logger() // logger.Info(). // Timestamp(). // Msg("dup") // // Output: {"level":"info","time":1494567715,"time":1494567715,"message":"dup"} // // In this case, many consumers will take the last value, // but this is not guaranteed; check yours if in doubt. // // Concurrency safety: // // Be careful when calling UpdateContext. It is not concurrency safe. Use the With method to create a child logger: // // func handler(w http.ResponseWriter, r *http.Request) { // // Create a child logger for concurrency safety // logger := log.Logger.With().Logger() // // // Add context fields, for example User-Agent from HTTP headers // logger.UpdateContext(func(c zerolog.Context) zerolog.Context { // ... // }) // } package zerolog import ( "context" "errors" "fmt" "io" "os" "strconv" "strings" ) // Level defines log levels. type Level int8 const ( // DebugLevel defines debug log level. DebugLevel Level = iota // InfoLevel defines info log level. InfoLevel // WarnLevel defines warn log level. WarnLevel // ErrorLevel defines error log level. ErrorLevel // FatalLevel defines fatal log level. FatalLevel // PanicLevel defines panic log level. PanicLevel // NoLevel defines an absent log level. NoLevel // Disabled disables the logger. Disabled // TraceLevel defines trace log level. TraceLevel Level = -1 // Values less than TraceLevel are handled as numbers. ) func (l Level) String() string { switch l { case TraceLevel: return LevelTraceValue case DebugLevel: return LevelDebugValue case InfoLevel: return LevelInfoValue case WarnLevel: return LevelWarnValue case ErrorLevel: return LevelErrorValue case FatalLevel: return LevelFatalValue case PanicLevel: return LevelPanicValue case Disabled: return "disabled" case NoLevel: return "" } return strconv.Itoa(int(l)) } // ParseLevel converts a level string into a zerolog Level value. // returns an error if the input string does not match known values. func ParseLevel(levelStr string) (Level, error) { switch { case strings.EqualFold(levelStr, LevelFieldMarshalFunc(TraceLevel)): return TraceLevel, nil case strings.EqualFold(levelStr, LevelFieldMarshalFunc(DebugLevel)): return DebugLevel, nil case strings.EqualFold(levelStr, LevelFieldMarshalFunc(InfoLevel)): return InfoLevel, nil case strings.EqualFold(levelStr, LevelFieldMarshalFunc(WarnLevel)): return WarnLevel, nil case strings.EqualFold(levelStr, LevelFieldMarshalFunc(ErrorLevel)): return ErrorLevel, nil case strings.EqualFold(levelStr, LevelFieldMarshalFunc(FatalLevel)): return FatalLevel, nil case strings.EqualFold(levelStr, LevelFieldMarshalFunc(PanicLevel)): return PanicLevel, nil case strings.EqualFold(levelStr, LevelFieldMarshalFunc(Disabled)): return Disabled, nil case strings.EqualFold(levelStr, LevelFieldMarshalFunc(NoLevel)): return NoLevel, nil } i, err := strconv.Atoi(levelStr) if err != nil { return NoLevel, fmt.Errorf("Unknown Level String: '%s', defaulting to NoLevel", levelStr) } if i > 127 || i < -128 { return NoLevel, fmt.Errorf("Out-Of-Bounds Level: '%d', defaulting to NoLevel", i) } return Level(i), nil } // UnmarshalText implements encoding.TextUnmarshaler to allow for easy reading from toml/yaml/json formats func (l *Level) UnmarshalText(text []byte) error { if l == nil { return errors.New("can't unmarshal a nil *Level") } var err error *l, err = ParseLevel(string(text)) return err } // MarshalText implements encoding.TextMarshaler to allow for easy writing into toml/yaml/json formats func (l Level) MarshalText() ([]byte, error) { return []byte(LevelFieldMarshalFunc(l)), nil } // A Logger represents an active logging object that generates lines // of JSON output to an io.Writer. Each logging operation makes a single // call to the Writer's Write method. There is no guarantee on access // serialization to the Writer. If your Writer is not thread safe, // you may consider a sync wrapper. type Logger struct { w LevelWriter level Level sampler Sampler context []byte hooks []Hook stack bool ctx context.Context } // New creates a root logger with given output writer. If the output writer implements // the LevelWriter interface, the WriteLevel method will be called instead of the Write // one. // // Each logging operation makes a single call to the Writer's Write method. There is no // guarantee on access serialization to the Writer. If your Writer is not thread safe, // you may consider using sync wrapper. func New(w io.Writer) Logger { if w == nil { w = io.Discard } lw, ok := w.(LevelWriter) if !ok { lw = LevelWriterAdapter{w} } return Logger{w: lw, level: TraceLevel} } // Nop returns a disabled logger for which all operation are no-op. func Nop() Logger { return New(nil).Level(Disabled) } // Output duplicates the current logger and sets w as its output. func (l Logger) Output(w io.Writer) Logger { l2 := New(w) l2.level = l.level l2.sampler = l.sampler l2.stack = l.stack if len(l.hooks) > 0 { l2.hooks = append(l2.hooks, l.hooks...) } if l.context != nil { l2.context = make([]byte, len(l.context), cap(l.context)) copy(l2.context, l.context) } return l2 } // With creates a child logger with the field added to its context. func (l Logger) With() Context { context := l.context l.context = make([]byte, 0, 500) if context != nil { l.context = append(l.context, context...) } else { // This is needed for AppendKey to not check len of input // thus making it inlinable l.context = enc.AppendBeginMarker(l.context) } return Context{l} } // UpdateContext updates the internal logger's context. // // Caution: This method is not concurrency safe. // Use the With method to create a child logger before modifying the context from concurrent goroutines. func (l *Logger) UpdateContext(update func(c Context) Context) { if l == disabledLogger { return } if cap(l.context) == 0 { l.context = make([]byte, 0, 500) } if len(l.context) == 0 { l.context = enc.AppendBeginMarker(l.context) } c := update(Context{*l}) l.context = c.l.context } // Level creates a child logger with the minimum accepted level set to level. func (l Logger) Level(lvl Level) Logger { l.level = lvl return l } // GetLevel returns the current Level of l. func (l Logger) GetLevel() Level { return l.level } // Sample returns a logger with the s sampler. func (l Logger) Sample(s Sampler) Logger { l.sampler = s return l } // Hook returns a logger with the h Hook. func (l Logger) Hook(hooks ...Hook) Logger { if len(hooks) == 0 { return l } newHooks := make([]Hook, len(l.hooks), len(l.hooks)+len(hooks)) copy(newHooks, l.hooks) l.hooks = append(newHooks, hooks...) return l } // Trace starts a new message with trace level. // // You must call Msg on the returned event in order to send the event. func (l *Logger) Trace() *Event { return l.newEvent(TraceLevel, nil) } // Debug starts a new message with debug level. // // You must call Msg on the returned event in order to send the event. func (l *Logger) Debug() *Event { return l.newEvent(DebugLevel, nil) } // Info starts a new message with info level. // // You must call Msg on the returned event in order to send the event. func (l *Logger) Info() *Event { return l.newEvent(InfoLevel, nil) } // Warn starts a new message with warn level. // // You must call Msg on the returned event in order to send the event. func (l *Logger) Warn() *Event { return l.newEvent(WarnLevel, nil) } // Error starts a new message with error level. // // You must call Msg on the returned event in order to send the event. func (l *Logger) Error() *Event { return l.newEvent(ErrorLevel, nil) } // Err starts a new message with error level with err as a field if not nil or // with info level if err is nil. // // You must call Msg on the returned event in order to send the event. func (l *Logger) Err(err error) *Event { if err != nil { return l.Error().Err(err) } return l.Info() } // Fatal starts a new message with fatal level. The os.Exit(1) function // is called by the Msg method, which terminates the program immediately. // // You must call Msg on the returned event in order to send the event. func (l *Logger) Fatal() *Event { return l.newEvent(FatalLevel, func(msg string) { if closer, ok := l.w.(io.Closer); ok { // Close the writer to flush any buffered message. Otherwise the message // will be lost as os.Exit() terminates the program immediately. closer.Close() } os.Exit(1) }) } // Panic starts a new message with panic level. The panic() function // is called by the Msg method, which stops the ordinary flow of a goroutine. // // You must call Msg on the returned event in order to send the event. func (l *Logger) Panic() *Event { return l.newEvent(PanicLevel, func(msg string) { panic(msg) }) } // WithLevel starts a new message with level. Unlike Fatal and Panic // methods, WithLevel does not terminate the program or stop the ordinary // flow of a goroutine when used with their respective levels. // // You must call Msg on the returned event in order to send the event. func (l *Logger) WithLevel(level Level) *Event { switch level { case TraceLevel: return l.Trace() case DebugLevel: return l.Debug() case InfoLevel: return l.Info() case WarnLevel: return l.Warn() case ErrorLevel: return l.Error() case FatalLevel: return l.newEvent(FatalLevel, nil) case PanicLevel: return l.newEvent(PanicLevel, nil) case NoLevel: return l.Log() case Disabled: return nil default: return l.newEvent(level, nil) } } // Log starts a new message with no level. Setting GlobalLevel to Disabled // will still disable events produced by this method. // // You must call Msg on the returned event in order to send the event. func (l *Logger) Log() *Event { return l.newEvent(NoLevel, nil) } // Print sends a log event using debug level and no extra field. // Arguments are handled in the manner of fmt.Print. func (l *Logger) Print(v ...interface{}) { if e := l.Debug(); e.Enabled() { e.CallerSkipFrame(1).Msg(fmt.Sprint(v...)) } } // Printf sends a log event using debug level and no extra field. // Arguments are handled in the manner of fmt.Printf. func (l *Logger) Printf(format string, v ...interface{}) { if e := l.Debug(); e.Enabled() { e.CallerSkipFrame(1).Msg(fmt.Sprintf(format, v...)) } } // Println sends a log event using debug level and no extra field. // Arguments are handled in the manner of fmt.Println. func (l *Logger) Println(v ...interface{}) { if e := l.Debug(); e.Enabled() { e.CallerSkipFrame(1).Msg(fmt.Sprintln(v...)) } } // Write implements the io.Writer interface. This is useful to set as a writer // for the standard library log. func (l Logger) Write(p []byte) (n int, err error) { n = len(p) if n > 0 && p[n-1] == '\n' { // Trim CR added by stdlog. p = p[0 : n-1] } l.Log().CallerSkipFrame(1).Msg(string(p)) return } func (l *Logger) newEvent(level Level, done func(string)) *Event { enabled := l.should(level) if !enabled { if done != nil { done("") } return nil } e := newEvent(l.w, level) e.done = done e.ch = l.hooks e.ctx = l.ctx if level != NoLevel && LevelFieldName != "" { e.Str(LevelFieldName, LevelFieldMarshalFunc(level)) } if l.context != nil && len(l.context) > 1 { e.buf = enc.AppendObjectData(e.buf, l.context) } if l.stack { e.Stack() } return e } // should returns true if the log event should be logged. func (l *Logger) should(lvl Level) bool { if l.w == nil { return false } if lvl < l.level || lvl < GlobalLevel() { return false } if l.sampler != nil && !samplingDisabled() { return l.sampler.Sample(lvl) } return true }
// Package log provides a global logger for zerolog. package log import ( "context" "fmt" "io" "os" "github.com/rs/zerolog" ) // Logger is the global logger. var Logger = zerolog.New(os.Stderr).With().Timestamp().Logger() // Output duplicates the global logger and sets w as its output. func Output(w io.Writer) zerolog.Logger { return Logger.Output(w) } // With creates a child logger with the field added to its context. func With() zerolog.Context { return Logger.With() } // Level creates a child logger with the minimum accepted level set to level. func Level(level zerolog.Level) zerolog.Logger { return Logger.Level(level) } // Sample returns a logger with the s sampler. func Sample(s zerolog.Sampler) zerolog.Logger { return Logger.Sample(s) } // Hook returns a logger with the h Hook. func Hook(h zerolog.Hook) zerolog.Logger { return Logger.Hook(h) } // Err starts a new message with error level with err as a field if not nil or // with info level if err is nil. // // You must call Msg on the returned event in order to send the event. func Err(err error) *zerolog.Event { return Logger.Err(err) } // Trace starts a new message with trace level. // // You must call Msg on the returned event in order to send the event. func Trace() *zerolog.Event { return Logger.Trace() } // Debug starts a new message with debug level. // // You must call Msg on the returned event in order to send the event. func Debug() *zerolog.Event { return Logger.Debug() } // Info starts a new message with info level. // // You must call Msg on the returned event in order to send the event. func Info() *zerolog.Event { return Logger.Info() } // Warn starts a new message with warn level. // // You must call Msg on the returned event in order to send the event. func Warn() *zerolog.Event { return Logger.Warn() } // Error starts a new message with error level. // // You must call Msg on the returned event in order to send the event. func Error() *zerolog.Event { return Logger.Error() } // Fatal starts a new message with fatal level. The os.Exit(1) function // is called by the Msg method. // // You must call Msg on the returned event in order to send the event. func Fatal() *zerolog.Event { return Logger.Fatal() } // Panic starts a new message with panic level. The message is also sent // to the panic function. // // You must call Msg on the returned event in order to send the event. func Panic() *zerolog.Event { return Logger.Panic() } // WithLevel starts a new message with level. // // You must call Msg on the returned event in order to send the event. func WithLevel(level zerolog.Level) *zerolog.Event { return Logger.WithLevel(level) } // Log starts a new message with no level. Setting zerolog.GlobalLevel to // zerolog.Disabled will still disable events produced by this method. // // You must call Msg on the returned event in order to send the event. func Log() *zerolog.Event { return Logger.Log() } // Print sends a log event using debug level and no extra field. // Arguments are handled in the manner of fmt.Print. func Print(v ...interface{}) { Logger.Debug().CallerSkipFrame(1).Msg(fmt.Sprint(v...)) } // Printf sends a log event using debug level and no extra field. // Arguments are handled in the manner of fmt.Printf. func Printf(format string, v ...interface{}) { Logger.Debug().CallerSkipFrame(1).Msgf(format, v...) } // Ctx returns the Logger associated with the ctx. If no logger // is associated, a disabled logger is returned. func Ctx(ctx context.Context) *zerolog.Logger { return zerolog.Ctx(ctx) }
package pkgerrors import ( "github.com/pkg/errors" ) var ( StackSourceFileName = "source" StackSourceLineName = "line" StackSourceFunctionName = "func" ) type state struct { b []byte } // Write implement fmt.Formatter interface. func (s *state) Write(b []byte) (n int, err error) { s.b = b return len(b), nil } // Width implement fmt.Formatter interface. func (s *state) Width() (wid int, ok bool) { return 0, false } // Precision implement fmt.Formatter interface. func (s *state) Precision() (prec int, ok bool) { return 0, false } // Flag implement fmt.Formatter interface. func (s *state) Flag(c int) bool { return false } func frameField(f errors.Frame, s *state, c rune) string { f.Format(s, c) return string(s.b) } // MarshalStack implements pkg/errors stack trace marshaling. // // zerolog.ErrorStackMarshaler = MarshalStack func MarshalStack(err error) interface{} { type stackTracer interface { StackTrace() errors.StackTrace } var sterr stackTracer var ok bool for err != nil { sterr, ok = err.(stackTracer) if ok { break } u, ok := err.(interface { Unwrap() error }) if !ok { return nil } err = u.Unwrap() } if sterr == nil { return nil } st := sterr.StackTrace() s := &state{} out := make([]map[string]string, 0, len(st)) for _, frame := range st { out = append(out, map[string]string{ StackSourceFileName: frameField(frame, s, 's'), StackSourceLineName: frameField(frame, s, 'd'), StackSourceFunctionName: frameField(frame, s, 'n'), }) } return out }
package zerolog import ( "math/rand" "sync/atomic" "time" ) var ( // Often samples log every ~ 10 events. Often = RandomSampler(10) // Sometimes samples log every ~ 100 events. Sometimes = RandomSampler(100) // Rarely samples log every ~ 1000 events. Rarely = RandomSampler(1000) ) // Sampler defines an interface to a log sampler. type Sampler interface { // Sample returns true if the event should be part of the sample, false if // the event should be dropped. Sample(lvl Level) bool } // RandomSampler use a PRNG to randomly sample an event out of N events, // regardless of their level. type RandomSampler uint32 // Sample implements the Sampler interface. func (s RandomSampler) Sample(lvl Level) bool { if s <= 0 { return false } if rand.Intn(int(s)) != 0 { return false } return true } // BasicSampler is a sampler that will send every Nth events, regardless of // their level. type BasicSampler struct { N uint32 counter uint32 } // Sample implements the Sampler interface. func (s *BasicSampler) Sample(lvl Level) bool { n := s.N if n == 0 { return false } if n == 1 { return true } c := atomic.AddUint32(&s.counter, 1) return c%n == 1 } // BurstSampler lets Burst events pass per Period then pass the decision to // NextSampler. If Sampler is not set, all subsequent events are rejected. type BurstSampler struct { // Burst is the maximum number of event per period allowed before calling // NextSampler. Burst uint32 // Period defines the burst period. If 0, NextSampler is always called. Period time.Duration // NextSampler is the sampler used after the burst is reached. If nil, // events are always rejected after the burst. NextSampler Sampler counter uint32 resetAt int64 } // Sample implements the Sampler interface. func (s *BurstSampler) Sample(lvl Level) bool { if s.Burst > 0 && s.Period > 0 { if s.inc() <= s.Burst { return true } } if s.NextSampler == nil { return false } return s.NextSampler.Sample(lvl) } func (s *BurstSampler) inc() uint32 { now := TimestampFunc().UnixNano() resetAt := atomic.LoadInt64(&s.resetAt) var c uint32 if now > resetAt { c = 1 atomic.StoreUint32(&s.counter, c) newResetAt := now + s.Period.Nanoseconds() reset := atomic.CompareAndSwapInt64(&s.resetAt, resetAt, newResetAt) if !reset { // Lost the race with another goroutine trying to reset. c = atomic.AddUint32(&s.counter, 1) } } else { c = atomic.AddUint32(&s.counter, 1) } return c } // LevelSampler applies a different sampler for each level. type LevelSampler struct { TraceSampler, DebugSampler, InfoSampler, WarnSampler, ErrorSampler Sampler } func (s LevelSampler) Sample(lvl Level) bool { switch lvl { case TraceLevel: if s.TraceSampler != nil { return s.TraceSampler.Sample(lvl) } case DebugLevel: if s.DebugSampler != nil { return s.DebugSampler.Sample(lvl) } case InfoLevel: if s.InfoSampler != nil { return s.InfoSampler.Sample(lvl) } case WarnLevel: if s.WarnSampler != nil { return s.WarnSampler.Sample(lvl) } case ErrorLevel: if s.ErrorSampler != nil { return s.ErrorSampler.Sample(lvl) } } return true }
// +build !windows // +build !binary_log package zerolog import ( "io" ) // See http://cee.mitre.org/language/1.0-beta1/clt.html#syslog // or https://www.rsyslog.com/json-elasticsearch/ const ceePrefix = "@cee:" // SyslogWriter is an interface matching a syslog.Writer struct. type SyslogWriter interface { io.Writer Debug(m string) error Info(m string) error Warning(m string) error Err(m string) error Emerg(m string) error Crit(m string) error } type syslogWriter struct { w SyslogWriter prefix string } // SyslogLevelWriter wraps a SyslogWriter and call the right syslog level // method matching the zerolog level. func SyslogLevelWriter(w SyslogWriter) LevelWriter { return syslogWriter{w, ""} } // SyslogCEEWriter wraps a SyslogWriter with a SyslogLevelWriter that adds a // MITRE CEE prefix for JSON syslog entries, compatible with rsyslog // and syslog-ng JSON logging support. // See https://www.rsyslog.com/json-elasticsearch/ func SyslogCEEWriter(w SyslogWriter) LevelWriter { return syslogWriter{w, ceePrefix} } func (sw syslogWriter) Write(p []byte) (n int, err error) { var pn int if sw.prefix != "" { pn, err = sw.w.Write([]byte(sw.prefix)) if err != nil { return pn, err } } n, err = sw.w.Write(p) return pn + n, err } // WriteLevel implements LevelWriter interface. func (sw syslogWriter) WriteLevel(level Level, p []byte) (n int, err error) { switch level { case TraceLevel: case DebugLevel: err = sw.w.Debug(sw.prefix + string(p)) case InfoLevel: err = sw.w.Info(sw.prefix + string(p)) case WarnLevel: err = sw.w.Warning(sw.prefix + string(p)) case ErrorLevel: err = sw.w.Err(sw.prefix + string(p)) case FatalLevel: err = sw.w.Emerg(sw.prefix + string(p)) case PanicLevel: err = sw.w.Crit(sw.prefix + string(p)) case NoLevel: err = sw.w.Info(sw.prefix + string(p)) default: panic("invalid level") } // Any CEE prefix is not part of the message, so we don't include its length n = len(p) return } // Call the underlying writer's Close method if it is an io.Closer. Otherwise // does nothing. func (sw syslogWriter) Close() error { if c, ok := sw.w.(io.Closer); ok { return c.Close() } return nil }
package zerolog import ( "bytes" "io" "path" "runtime" "strconv" "strings" "sync" ) // LevelWriter defines as interface a writer may implement in order // to receive level information with payload. type LevelWriter interface { io.Writer WriteLevel(level Level, p []byte) (n int, err error) } // LevelWriterAdapter adapts an io.Writer to support the LevelWriter interface. type LevelWriterAdapter struct { io.Writer } // WriteLevel simply writes everything to the adapted writer, ignoring the level. func (lw LevelWriterAdapter) WriteLevel(l Level, p []byte) (n int, err error) { return lw.Write(p) } // Call the underlying writer's Close method if it is an io.Closer. Otherwise // does nothing. func (lw LevelWriterAdapter) Close() error { if closer, ok := lw.Writer.(io.Closer); ok { return closer.Close() } return nil } type syncWriter struct { mu sync.Mutex lw LevelWriter } // SyncWriter wraps w so that each call to Write is synchronized with a mutex. // This syncer can be used to wrap the call to writer's Write method if it is // not thread safe. Note that you do not need this wrapper for os.File Write // operations on POSIX and Windows systems as they are already thread-safe. func SyncWriter(w io.Writer) io.Writer { if lw, ok := w.(LevelWriter); ok { return &syncWriter{lw: lw} } return &syncWriter{lw: LevelWriterAdapter{w}} } // Write implements the io.Writer interface. func (s *syncWriter) Write(p []byte) (n int, err error) { s.mu.Lock() defer s.mu.Unlock() return s.lw.Write(p) } // WriteLevel implements the LevelWriter interface. func (s *syncWriter) WriteLevel(l Level, p []byte) (n int, err error) { s.mu.Lock() defer s.mu.Unlock() return s.lw.WriteLevel(l, p) } func (s *syncWriter) Close() error { s.mu.Lock() defer s.mu.Unlock() if closer, ok := s.lw.(io.Closer); ok { return closer.Close() } return nil } type multiLevelWriter struct { writers []LevelWriter } func (t multiLevelWriter) Write(p []byte) (n int, err error) { for _, w := range t.writers { if _n, _err := w.Write(p); err == nil { n = _n if _err != nil { err = _err } else if _n != len(p) { err = io.ErrShortWrite } } } return n, err } func (t multiLevelWriter) WriteLevel(l Level, p []byte) (n int, err error) { for _, w := range t.writers { if _n, _err := w.WriteLevel(l, p); err == nil { n = _n if _err != nil { err = _err } else if _n != len(p) { err = io.ErrShortWrite } } } return n, err } // Calls close on all the underlying writers that are io.Closers. If any of the // Close methods return an error, the remainder of the closers are not closed // and the error is returned. func (t multiLevelWriter) Close() error { for _, w := range t.writers { if closer, ok := w.(io.Closer); ok { if err := closer.Close(); err != nil { return err } } } return nil } // MultiLevelWriter creates a writer that duplicates its writes to all the // provided writers, similar to the Unix tee(1) command. If some writers // implement LevelWriter, their WriteLevel method will be used instead of Write. func MultiLevelWriter(writers ...io.Writer) LevelWriter { lwriters := make([]LevelWriter, 0, len(writers)) for _, w := range writers { if lw, ok := w.(LevelWriter); ok { lwriters = append(lwriters, lw) } else { lwriters = append(lwriters, LevelWriterAdapter{w}) } } return multiLevelWriter{lwriters} } // TestingLog is the logging interface of testing.TB. type TestingLog interface { Log(args ...interface{}) Logf(format string, args ...interface{}) Helper() } // TestWriter is a writer that writes to testing.TB. type TestWriter struct { T TestingLog // Frame skips caller frames to capture the original file and line numbers. Frame int } // NewTestWriter creates a writer that logs to the testing.TB. func NewTestWriter(t TestingLog) TestWriter { return TestWriter{T: t} } // Write to testing.TB. func (t TestWriter) Write(p []byte) (n int, err error) { t.T.Helper() n = len(p) // Strip trailing newline because t.Log always adds one. p = bytes.TrimRight(p, "\n") // Try to correct the log file and line number to the caller. if t.Frame > 0 { _, origFile, origLine, _ := runtime.Caller(1) _, frameFile, frameLine, ok := runtime.Caller(1 + t.Frame) if ok { erase := strings.Repeat("\b", len(path.Base(origFile))+len(strconv.Itoa(origLine))+3) t.T.Logf("%s%s:%d: %s", erase, path.Base(frameFile), frameLine, p) return n, err } } t.T.Log(string(p)) return n, err } // ConsoleTestWriter creates an option that correctly sets the file frame depth for testing.TB log. func ConsoleTestWriter(t TestingLog) func(w *ConsoleWriter) { return func(w *ConsoleWriter) { w.Out = TestWriter{T: t, Frame: 6} } } // FilteredLevelWriter writes only logs at Level or above to Writer. // // It should be used only in combination with MultiLevelWriter when you // want to write to multiple destinations at different levels. Otherwise // you should just set the level on the logger and filter events early. // When using MultiLevelWriter then you set the level on the logger to // the lowest of the levels you use for writers. type FilteredLevelWriter struct { Writer LevelWriter Level Level } // Write writes to the underlying Writer. func (w *FilteredLevelWriter) Write(p []byte) (int, error) { return w.Writer.Write(p) } // WriteLevel calls WriteLevel of the underlying Writer only if the level is equal // or above the Level. func (w *FilteredLevelWriter) WriteLevel(level Level, p []byte) (int, error) { if level >= w.Level { return w.Writer.WriteLevel(level, p) } return len(p), nil } var triggerWriterPool = &sync.Pool{ New: func() interface{} { return bytes.NewBuffer(make([]byte, 0, 1024)) }, } // TriggerLevelWriter buffers log lines at the ConditionalLevel or below // until a trigger level (or higher) line is emitted. Log lines with level // higher than ConditionalLevel are always written out to the destination // writer. If trigger never happens, buffered log lines are never written out. // // It can be used to configure "log level per request". type TriggerLevelWriter struct { // Destination writer. If LevelWriter is provided (usually), its WriteLevel is used // instead of Write. io.Writer // ConditionalLevel is the level (and below) at which lines are buffered until // a trigger level (or higher) line is emitted. Usually this is set to DebugLevel. ConditionalLevel Level // TriggerLevel is the lowest level that triggers the sending of the conditional // level lines. Usually this is set to ErrorLevel. TriggerLevel Level buf *bytes.Buffer triggered bool mu sync.Mutex } func (w *TriggerLevelWriter) WriteLevel(l Level, p []byte) (n int, err error) { w.mu.Lock() defer w.mu.Unlock() // At first trigger level or above log line, we flush the buffer and change the // trigger state to triggered. if !w.triggered && l >= w.TriggerLevel { err := w.trigger() if err != nil { return 0, err } } // Unless triggered, we buffer everything at and below ConditionalLevel. if !w.triggered && l <= w.ConditionalLevel { if w.buf == nil { w.buf = triggerWriterPool.Get().(*bytes.Buffer) } // We prefix each log line with a byte with the level. // Hopefully we will never have a level value which equals a newline // (which could interfere with reconstruction of log lines in the trigger method). w.buf.WriteByte(byte(l)) w.buf.Write(p) return len(p), nil } // Anything above ConditionalLevel is always passed through. // Once triggered, everything is passed through. if lw, ok := w.Writer.(LevelWriter); ok { return lw.WriteLevel(l, p) } return w.Write(p) } // trigger expects lock to be held. func (w *TriggerLevelWriter) trigger() error { if w.triggered { return nil } w.triggered = true if w.buf == nil { return nil } p := w.buf.Bytes() for len(p) > 0 { // We do not use bufio.Scanner here because we already have full buffer // in the memory and we do not want extra copying from the buffer to // scanner's token slice, nor we want to hit scanner's token size limit, // and we also want to preserve newlines. i := bytes.IndexByte(p, '\n') line := p[0 : i+1] p = p[i+1:] // We prefixed each log line with a byte with the level. level := Level(line[0]) line = line[1:] var err error if lw, ok := w.Writer.(LevelWriter); ok { _, err = lw.WriteLevel(level, line) } else { _, err = w.Write(line) } if err != nil { return err } } return nil } // Trigger forces flushing the buffer and change the trigger state to // triggered, if the writer has not already been triggered before. func (w *TriggerLevelWriter) Trigger() error { w.mu.Lock() defer w.mu.Unlock() return w.trigger() } // Close closes the writer and returns the buffer to the pool. func (w *TriggerLevelWriter) Close() error { w.mu.Lock() defer w.mu.Unlock() if w.buf == nil { return nil } // We return the buffer only if it has not grown above the limit. // This prevents accumulation of large buffers in the pool just // because occasionally a large buffer might be needed. if w.buf.Cap() <= TriggerLevelWriterBufferReuseLimit { w.buf.Reset() triggerWriterPool.Put(w.buf) } w.buf = nil return nil }