Ask a Question
GoPro Support Hub Ask a question. Share an answer. Find a solution. Stay stoked.
Announcements
Is your GoPro gear up to date? Check to see If it is on our Update page.
Cameras
Reply
Highlighted
Tourist
Posts: 4

Re: Combine Several video captures and retain GPS data

It wasn't but your instincts are right. Embrassingly, I had run out of space on that disk - now humming along happily again :)

 

thanks for responding

 

Sam

Highlighted
Sightseer
Posts: 1

Re: Combine Several video captures and retain GPS data

I tried to merge my csv files generated from DashWare, but the merged file only contains time data, nothing else. Any ideas?

defger.jpg
Highlighted
Tourist
Posts: 14

Re: Combine Several video captures and retain GPS data

[ Edited ]

Send me your CSV files and I will take a look. PM sent with contact information.

Highlighted
Tourist
Posts: 5

Re: Combine Several video captures and retain GPS data

After a deeper look I think I found the reason for the issues with the data merging tools currently available. I tried using both the "GoProDataMerge" and "GoProCsvMerger" tool by @marce4453 found earlier in this thread here: https://community.gopro.com/t5/Cameras/Combine-Several-video-captures-and-retain-GPS-data/m-p/287975...

 

1) A problem with decimal points in the data, using different language/localization/number formatting settings in the Operating System for users outside the USA (e.g. I am using nl-NL instead of en-US) --> the first file will go fine, but as soon as the second file gets merged and calculations (cumulation of last GlobalTime on top of current file GlobalTime) then the decimals get removed while parsing the string. In the source code when using e.g. C# the decimal.Parse() method would need a second parameter specifying e.g. new CultureInfo("en-US").NumberFormat to explicitly use "." as decimal separator instead of e.g. "," when parsing, because Dashware's files are using format "x.000000" instead of e.g. "x,0000000" (period instead of comma).

 

For example, the GlobalTime in the first file will be exported as "530.950000", while the next line (coming from the second file) will be formatted as "530950000" (note the missing decimal separator).

 

2) The other issue is that different files in the same set (e.g. GH010046_out.csv and GH020046_out.csv) can have different sortings/orders of the header columns. So the merging tools currently have no sorting logic, meaning that values can get mixed up randomly. For example, not the last colum of both files in the same recording session/data set ("Hero7 Black:TIMO[0]" vs "Hero7 Black:SROT[0]", while both files contain the same columns in the end but in a different order): 

 

Headers in example file part 1 "GH010046_out.csv":

GlobalTime,Hero7 Black:TSMP[Time],Hero7 Black:TSMP[0],Hero7 Black:SIUN[Time],Hero7 Black:SIUN[0],Hero7 Black:SIUN[1],Hero7 Black:SIUN[2],Hero7 Black:SIUN[3],Hero7 Black:MTRX[Time],Hero7 Black:MTRX[0],Hero7 Black:MTRX[1],Hero7 Black:MTRX[2],Hero7 Black:MTRX[3],Hero7 Black:MTRX[4],Hero7 Black:MTRX[5],Hero7 Black:MTRX[6],Hero7 Black:MTRX[7],Hero7 Black:MTRX[8],Hero7 Black:ORIN[Time],Hero7 Black:ORIN[0],Hero7 Black:ORIO[Time],Hero7 Black:ORIO[0],Hero7 Black:ACCL[Time],Hero7 Black:ACCL[0],Hero7 Black:ACCL[1],Hero7 Black:ACCL[2],Hero7 Black:GYRO[Time],Hero7 Black:GYRO[0],Hero7 Black:GYRO[1],Hero7 Black:GYRO[2],Hero7 Black:GPSF[Time],Hero7 Black:GPSF[0],Hero7 Black:GPSU[Time],Hero7 Black:GPSU[0],Hero7 Black:GPSP[Time],Hero7 Black:GPSP[0],Hero7 Black:GPS5[Time],Hero7 Black:GPS5[0] (deg),Hero7 Black:GPS5[1] (deg),Hero7 Black:GPS5[2] (m),Hero7 Black:GPS5[3] (m/s),Hero7 Black:GPS5[4] (m/s),Hero7 Black:ISOE[Time],Hero7 Black:ISOE[0],Hero7 Black:SHUT[Time],Hero7 Black:SHUT[0],Hero7 Black:WBAL[Time],Hero7 Black:WBAL[0],Hero7 Black:WRGB[Time],Hero7 Black:WRGB[0],Hero7 Black:WRGB[1],Hero7 Black:WRGB[2],Hero7 Black:YAVG[Time],Hero7 Black:YAVG[0],Hero7 Black:UNIF[Time],Hero7 Black:UNIF[0],Hero7 Black:SCEN[Time],Hero7 Black:SCEN[0],Hero7 Black:SCEN[1],Hero7 Black:HUES[Time],Hero7 Black:HUES[0],Hero7 Black:HUES[1],Hero7 Black:SROT[Time],Hero7 Black:SROT[0],Hero7 Black:TIMO[Time],Hero7 Black:TIMO[0]

 

Headers in example file part 2 "GH020046_out.csv":

GlobalTime,Hero7 Black:TSMP[Time],Hero7 Black:TSMP[0],Hero7 Black:SIUN[Time],Hero7 Black:SIUN[0],Hero7 Black:SIUN[1],Hero7 Black:SIUN[2],Hero7 Black:SIUN[3],Hero7 Black:MTRX[Time],Hero7 Black:MTRX[0],Hero7 Black:MTRX[1],Hero7 Black:MTRX[2],Hero7 Black:MTRX[3],Hero7 Black:MTRX[4],Hero7 Black:MTRX[5],Hero7 Black:MTRX[6],Hero7 Black:MTRX[7],Hero7 Black:MTRX[8],Hero7 Black:ORIN[Time],Hero7 Black:ORIN[0],Hero7 Black:ORIO[Time],Hero7 Black:ORIO[0],Hero7 Black:ACCL[Time],Hero7 Black:ACCL[0],Hero7 Black:ACCL[1],Hero7 Black:ACCL[2],Hero7 Black:GYRO[Time],Hero7 Black:GYRO[0],Hero7 Black:GYRO[1],Hero7 Black:GYRO[2],Hero7 Black:GPSF[Time],Hero7 Black:GPSF[0],Hero7 Black:GPSU[Time],Hero7 Black:GPSU[0],Hero7 Black:GPSP[Time],Hero7 Black:GPSP[0],Hero7 Black:GPS5[Time],Hero7 Black:GPS5[0] (deg),Hero7 Black:GPS5[1] (deg),Hero7 Black:GPS5[2] (m),Hero7 Black:GPS5[3] (m/s),Hero7 Black:GPS5[4] (m/s),Hero7 Black:TIMO[Time],Hero7 Black:TIMO[0],Hero7 Black:ISOE[Time],Hero7 Black:ISOE[0],Hero7 Black:SHUT[Time],Hero7 Black:SHUT[0],Hero7 Black:WBAL[Time],Hero7 Black:WBAL[0],Hero7 Black:WRGB[Time],Hero7 Black:WRGB[0],Hero7 Black:WRGB[1],Hero7 Black:WRGB[2],Hero7 Black:YAVG[Time],Hero7 Black:YAVG[0],Hero7 Black:UNIF[Time],Hero7 Black:UNIF[0],Hero7 Black:SCEN[Time],Hero7 Black:SCEN[0],Hero7 Black:SCEN[1],Hero7 Black:HUES[Time],Hero7 Black:HUES[0],Hero7 Black:HUES[1],Hero7 Black:SROT[Time],Hero7 Black:SROT[0]

 

Not sure why GoPro of Dashware would export different orders of the CSV columns, but I see this happening with my GoPro Hero 7 Black videos.

Highlighted
Tourist
Posts: 14

Re: Combine Several video captures and retain GPS data

That's a simple fix, I will update the app now and submit (they take a few days to approve and load to the site).

Highlighted
Tourist
Posts: 7

Re: Combine Several video captures and retain GPS data

Thanks @xfinity that's good analysis. I can only speak for my code but I was being lazy and not considering localisation at all. 

 

The column ordering one is interesting. Obviously the thing to do there is a full parse of the file to get the columns correct instead of just blindly appending the data in the order it came in. Shows how "lazy" comes back to bite you! (again, talking about myself here.). Although that does mean a reliance on the column names, which in turn requires it to understand the output of each specific camera rather than being generic. Annoying.

Highlighted
Tourist
Posts: 5

Re: Combine Several video captures and retain GPS data

For Dashware and other applications the order of the columns doesnt matter, so a quick and dirty solution could be to sort the record/row based on the key name (assuming each video in the recording session/set does include the same columns, which seems to be the case in all my videos)

 

when using CsvHelper this could be a quick-and-dirty fix (still being lazy :)) without requiring looping through all files first:

 

var sortedLastRow = ((IDictionary<string, object>)lastRow)
.Select(x => new { x.Key, x.Value })
.OrderBy(x => x.Key);

if (headersWritten == false)
{
sortedLastRow.ToList().ForEach(x =>
{
csvWriter.WriteField(x.Key);
});
csvWriter.NextRecord();
headersWritten = true;
}

sortedLastRow.ToList().ForEach(x =>
{
csvWriter.WriteField(x.Value);
});


//csvWriter.WriteRecord(lastRow); //Disabled this, added above logic to sort the columns
csvWriter.NextRecord();

Highlighted
Tourist
Posts: 14

Re: Combine Several video captures and retain GPS data

Column sequence isn't important with my app and it doesn't require exact names (although it could be improved).

 

I've fixed for localization and uploaded to "download.com" so it should be available later this week.