Gowhere – Read CSV file
Continue my Gowhere journey. I need to read data from a CSV file—a common task in programming, dealing with file.
A sample content with 3 columns ID, Effort, and Title.
ID,Effort,Title
"94503","4","Bug: The line was not aligned correctly"
"97018","12","Implement a cool feature"
"97595","1","Document an incident"The file size is small so I do not have to worry too much about the performance at this point. Let’s see what requires to perform the task in Go.
In general, here are steps to read and parse a CSV file
- Open the file
- Read everything or line by line
- Parse the text into desired outcome format
- Close the file. When to close depends on how you read the file
Open a file
Go supplies a built-in package "os" to work with the file system. Opening a file will return a pointer to the file—if no error, otherwise an error, a normal pattern in Go.
There are 3 required steps
- Open the file by
os.Open(file) - Check for error before usage
- Close the file by using a defer function. This is important otherwise the file is locked
// Need the os package to read open a file
// f is a pointer to File object (*File)
f, err := os.Open(file)
// Check for error before usage
if err != nil {
writeLog(err)
return nil
}
// Close file by a defer function
defer func() {
er := f.Close()
if er != nil {
writeLog(er)
}
}()Read file
Depending on the file size and the type of applications, developers can choose either read all contents at once or line by line. I always prefer line by line.
// This is a cool concept. Given that we have a pointer to the file opened by the os.
// A scanner is created and scan line by line
b := bufio.NewScanner(f)
stats := make(map[string]int64)
for b.Scan() {
line := b.Text()
parts := strings.Split(line, ",")
effortText := strings.Trim(parts[1], "\"")
if effortText == "" {
continue
}
effort, err := strconv.ParseInt(effortText, 0, 0)
if err != nil {
writeLog(err)
continue
}
id := strings.Trim(parts[0], "\"")
stats[id] = effort
fmt.Printf("%s - %d\n", id, effort)
}Go supplies bufio package to help manipulating IO (file). The first time I heard about the Scanner concept. It hits my mind immediately, kind of "wow that makes sense and cool".
After holding a point to a Scanner:
- Call
Scan()to loop throw the buffer data - Call
Text()to access the current line. If the file is opened in binary mode, use a different method - Parse the line to meet your logic
Proceed data
For common operations on a string, strings package is there. For conversion from a string to various data types—int, float, …—strconv package is there. They are all built-in packages.
Close the file
Handle by the defer function, which is called when the caller function exists.
// Close file by a defer function
defer func() {
er := f.Close()
if er != nil {
writeLog(er)
}
}()