How can I read lines from a file where the line endings are carriage return (CR), newline (NL), or both?
The PDF specification allows lines to end with CR, LF, or CRLF.
bufio.Reader.ReadString() and bufio.Reader.ReadBytes() allow a single delimiter byte.
bufio.Scanner.Scan() handles optionally preceded by , but not a lone .
The end-of-line marker is one optional carriage return followed by one mandatory newline.
Do I need to write my own function that uses bufio.Reader.ReadByte()?
You can write custom bufio.SplitFunc for bufio.Scanner. E.g:
// Mostly bufio.ScanLines code:
func ScanPDFLines(data []byte, atEOF bool) (advance int, token []byte, err error) {
if atEOF && len(data) == 0 {
return 0, nil, nil
}
if i := bytes.IndexAny(data, "
"); i >= 0 {
if data[i] == '
' {
// We have a line terminated by single newline.
return i + 1, data[0:i], nil
}
advance = i + 1
if len(data) > i+1 && data[i+1] == '
' {
advance += 1
}
return advance, data[0:i], nil
}
// If we're at EOF, we have a final, non-terminated line. Return it.
if atEOF {
return len(data), data, nil
}
// Request more data.
return 0, nil, nil
}
And use it like:
scan := bufio.NewScanner(r)
scan.Split(ScanPDFLines)