big-bro-watches-everything/devstar
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Properly determine CSV delimiter (#17459)

repo.diff.browse_source 
* Fixes #16558 CSV delimiter determiner

* Fixes #16558 - properly determine CSV delmiiter

* Moves quoteString to a new function

* Adds big test with lots of commas for tab delimited csv

* Adds comments

* Shortens the text of the test

* Removes single quotes from regexp as only double quotes need to be searched

* Fixes spelling

* Fixes check of length as it probalby will only be 1e4, not greater

* Makes sample size a const, properly removes truncated line

* Makes sample size a const, properly removes truncated line

* Fixes comment

* Fixes comment

* tests for FormatError() function

* Adds logic to find the limiter before or after a quoted value

* Simplifies regex

* Error tests

* Error tests

* Update modules/csv/csv.go

Co-authored-by: delvh <dev.lh@web.de>

* Update modules/csv/csv.go

Co-authored-by: delvh <dev.lh@web.de>

* Adds comments

* Update modules/csv/csv.go

Co-authored-by: delvh <dev.lh@web.de>

Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
Co-authored-by: zeripath <art27@cantab.net>
Co-authored-by: delvh <dev.lh@web.de>
This commit is contained in:
Richard Mahn
2021-10-30 09:50:40 -06:00
repo.diff.committed_by GitHub
repo.diff.parent 63c0dc89ef
repo.diff.commit 40c8451b7d
repo.diff.stats_desc%!(EXTRA int=5, int=642, int=94)
repo.diff.download_patch repo.diff.download_diff Expand all files Collapse all files

8
services/gitdiff/csv_test.go
repo.diff.view_file

@@ -194,16 +194,16 @@ c,d,e`,
var baseReader *csv.Reader
if len(c.base) > 0 {
baseReader, err = csv_module.CreateReaderAndGuessDelimiter(strings.NewReader(c.base))
baseReader, err = csv_module.CreateReaderAndDetermineDelimiter(nil, strings.NewReader(c.base))
if err != nil {
t.Errorf("CreateReaderAndGuessDelimiter failed: %s", err)
t.Errorf("CreateReaderAndDetermineDelimiter failed: %s", err)
}
}
var headReader *csv.Reader
if len(c.head) > 0 {
headReader, err = csv_module.CreateReaderAndGuessDelimiter(strings.NewReader(c.head))
headReader, err = csv_module.CreateReaderAndDetermineDelimiter(nil, strings.NewReader(c.head))
if err != nil {
t.Errorf("CreateReaderAndGuessDelimiter failed: %s", err)
t.Errorf("CreateReaderAndDetermineDelimiter failed: %s", err)
}
}