Ask a Question
GoPro Support Hub Ask a question. Share an answer. Find a solution. Stay stoked.
Announcements
Is your GoPro gear up to date? Check to see If it is on our Update page.
Cameras
Reply
Tourist
Posts: 1

Combine Several video captures and retain GPS data

[ New ]

I recorded about 1.5 hours of video and the camera broke them down to 17 minute files.  How do I combine them into 1 long video and retain the GPS data?  I would like to do the overlays and have them show as 1 contunious event, such as the path traveled as 1 complete circuit.

Thanks,

 

Thomas

GoPro
Posts: 19,608

Re: Combine Several video captures and retain GPS data

[ New ]

Hello @iflyppgnm

 

Checkout the articles below on using gauges in Quik for Desktop and overlaying telemetry in Dashware: 

 

  1. DASHWARE: https://gopro.com/help/articles/how_to/How-to-Overlay-Telemetry-from-GoPro-Cameras-in-Dashware
  2. QUIK: https://gopro.com/help/articles/block/How-to-Use-Gauges-in-Quik-for-Desktop

Let us know how it goes. 

 

Thanks, 
Marius

Tourist
Posts: 16

Re: Combine Several video captures and retain GPS data

[ New ]

@mariustanya no that doesnt answer anything.  Did you even read his question?  

 

Long recordings are split into shorter videos.  How do you merge the videos to have one continuous GPS track instead of a gps track limited to each split video?

Tourist
Posts: 16

Re: Combine Several video captures and retain GPS data

[ New ]

Hey i found a solution and wrote a blog post for everyone else.  Let me know if this works for yah.

 

GoPro really dropped the ball on their software and thats why everyone is leaving them.  Great cameras... horrible software

 

https://wordpress.com/post/hellomihai.wordpress.com/630

Tourist
Posts: 16

Re: Combine Several video captures and retain GPS data

[ New ]
Sightseer
Posts: 2

Re: Combine Several video captures and retain GPS data

[ New ]

I have tried the solution posted by hellomihai, but that didn't work for me: Dashware tells me it failed merging GPS data, it doesn't show the merged video, and it crashes if I try exporting the video.

 

I've tried RaceRender: no success.

 

I've tried MP4Joiner: no success.

 

I've tried Sony Vegas: no success.

 

GoPro Studio has been discontinued, so that's not an option.

 

Quik is what it is: a little tool for home-users to create short videos. Not what I need.

 

I am beyond frustrated with my GoPro Hero 7. The video quality is outstanding, no question about that. However, that is worthless for me. I use my camera for motorcycle track days. I want to have 1 video file per track session (roughly 20 minutes), containing gauges and track map powered by the camera's GPS data. I simply can't find an easy and reliable way to do that.

 

If I can't find a solution for this, I'll have to return this camera, which at the moment is useless for me. It really makes me regret having treaded in my Sena Prism to get this GoPro for $300. I'm very, very disappointed.

GoPro
Posts: 27,018

Re: Combine Several video captures and retain GPS data

[ New ]

 

Hi @redsand31551

 

I am sorry for any troubles caused and thank you for the information shared.

You may contact our Support Team for further assistance. https://gopro.com/help/ContactUs

If you wish to return the camera, please refer to the same store/channel you purchased it from.

 

Regards,

Ej

Adventurer
Posts: 13,858

Re: Combine Several video captures and retain GPS data

[ New ]

you have 1.5 hr of video 17 per chapter.... say 5 files..  go open quick go take chapter 1 and addwhat you like.  and render the file.   it creates   say gopro_0001, gopro_0001(01)    it does not over write the original.  so what you have is the original file and your overlay.  Understanding you want all 5 Files with one video and one shot overlay not making a mistake to be bigger or smaller or incorrectly positioned. that I understand.

 

 that will be the free way of doing it

 

Original does not get erased the  rendered gets overlays  all you need now to do is   addto your editor and be done guessing how it works.

Sightseer
Posts: 1

Re: Combine Several video captures and retain GPS data

[ New ]
Im in the same page, i race hare-scramble for 1.5 - 2hrs and i usually have more than 6 files about 4gb each, you can add speedometer and other stuff to each video then put them together, what we cant do and is my biggest problem is that we cant have is the whole gps path and info like total miles and thats exactly what i want, i think its something causing trouble to a lot of gopro customers, however i was able to do it once with dashware, i merge all the videos with gps data files and made 1 video for the whole race with the whole path and total miles, it was a pain because it took me like 6 hrs to make it.
Tourist
Posts: 8

Re: Combine Several video captures and retain GPS data

[ New ]

I'd like to know how people are handling this as well. I understand why it's split into 17 minute files, but why can't the software handle that gracefully? Surely it's just a matter of storing some metadata somewhere to track what files are part of a continus stream, then present it as such in the desktop environment? Give users the options to render out the single whole viedo with overlays?

GoPro
Posts: 27,018

Re: Combine Several video captures and retain GPS data

[ New ]

 

Hi @camerone60

 

Were you able to try the steps listed by fishycomics?

You may also try using RaceRender or Dahsware.

 

 

Thanks!

Ej

Tourist
Posts: 8

Re: Combine Several video captures and retain GPS data

[ New ]

Nope... I've tried Dashware, and it gives me an error "Data extraction/merge failed." when I first open the files for merging. It then manages to merge all the files into a single video, but loses all the GPS data. I can see there's CSV files extracts with this data, but even adding it in doesn't produce any data to be used in the overlays.

Tourist
Posts: 2

Re: Combine Several video captures and retain GPS data

[ New ]

I'm in the same boat, we need an option to record as a single file, my card is already formatted as exfat, I don't care about data corruption, either let us record as a single video or enable Quik to combine video files.

Tourist
Posts: 2

Re: Combine Several video captures and retain GPS data

[ New ]
I'm using Race Render for such videos.
It does the job almost perfectly! Combining all parts of the video and overlaying GPS data)

One thing is that it can't combine data from several videos (e.g. if you stopped and resumed recording again).
Tourist
Posts: 7

Re: Combine Several video captures and retain GPS data

[ New ]

 

I had the same problem recently, it's a massive PITA. The data is in the files but each file is self contained, so the full map and times aren't persisted across the files. (rather each file contains its own map segment and own start/end times from zero). 

 

I ended up writing a quick and dirty app to combine the Dashware CSV files into one data file, to be used with the merged video. You can't just append the files, you need to keep accumulating the numbers for time and distance so that it represents a single start-end run for the length of the combined video.

 

The app is nasty but it works. I've put it here if anyone wants it. It will work with the 6 black and 7 black, nothing else. 

 

https://www.dropbox.com/s/6y7qgich0ue4vij/GoPro%20CSV%20Merger.rar?dl=0

 

Basically:

 

1) (prerequisite) Put the hero 6 and 7 GPMF.XML profiles from the quick install into C:\Program Files\DashWare\DataTool\DataProfiles

2) Use dashware to merge the video files. It will complain about data extract / merge failed, but it will produce a merged video and it will extract the individual CSVs containing the telemetry data to the same folder

3) Use my app to merge the CSVs

4) Use the merged CSV as the data file for the merged video

 

You can then either export the merged video with telemetry gauges from dashware, or use dashware to make a green screen overlay of just the gagues.

 

I'm stunned at how crap the GoPro experience is. The file naming confention is idiotic, the telemetry is a mess and every time I think I'm turning the camera off I'm just switching it to photo mode. The hardware is pretty good though, I'll give it that.

Tourist
Posts: 3

Re: Combine Several video captures and retain GPS data

[ New ]

Hi,

 

thanks a lot for the programm, I'll try it this week and let you know.

 

Regards,

 

Julien

Tourist
Posts: 2

Re: Combine Several video captures and retain GPS data

[ New ]

I agree, hardware is good but software is crap, could you mod the script to work with the 5 Black?

Tourist
Posts: 8

Re: Combine Several video captures and retain GPS data

[ New ]

Interesting, so if you combine (and accumulate) the CSV files it generates, it will work?

 

I'll write up an app for myself if that's the case, as I've got a HERO 7 Silver, not Black... (I also found a script somewhere that extracts this data from the .MP4 file, using ffmpeg, maybe, just maybe, it would be possible to use ffmpeg to combine them all back together, including bring in the GPS metadata that has been properly incremented...)

 

As with everynone else, I find it utterly ridiculious that GoPro have such poor software, that we're resorting to writing our own programs to put back together the files their camera breaks up... If they're going to do that, at least have software to put it back together. Just pathetic really.

Tourist
Posts: 3

Re: Combine Several video captures and retain GPS data

[ New ]

Hi,

 

I tried the program to merge CSVs, and unfortunatly it does not work for me. The program well creates an output CSV file, but then I have error messages and the program crashes. The out put CSV file seems to be exactly the same CSV as the first I want to merge.

 

I tried with 2 and 3 CSV files, both crash :-(

 

I have a Hero black 6 and use windows 10

Tourist
Posts: 7

Re: Combine Several video captures and retain GPS data

[ New ]

The app was something I threw together for myself. If it works then great, if not then I'm sorry but I don't have the time to support it or make it work for other models. I can document what I did though, it may help other people.

 

First, you need to make sure that you have copied the Gopro GPMF profiles from the Quik install into the Dashware folder. Without this, Dashware is unable to parse / extract the telemetry properly. There are other posts on this forum about that, I don't have the link now but a quick search will find them.

 

When you look at the Dashware extracted CSV you will see something like 80 columns of data. Some of those are snapshots, eg GPS position and orientation. Some of them are accumulated, eg distance travelled or total descent. 

 

The trick when combining the telemetry is to identify the accumulated columns and add the value from the last row of the previous file to them when you are combining them. (actually add the last accumulated value, which you update at the end of every file so it keeps growing. You get the idea).

 

That's it - not a complicated problem to solve, but annoying to make universal because the different gopro models have different data with different column headings.

 

I'll dig out what I identified as the accumulating columns when I get some time later and make a separate post for them. 

Tourist
Posts: 7

Re: Combine Several video captures and retain GPS data

[ New ]
Here you go, these are the columns that I thought looked like accumulation columns. The other columns I just copied into the merged CSV unaltered.
 
"GlobalTime"
, "Total Ascent Meters", "Total Descent Meters", "Total Ascent Feet", "Total Descent Feet", "Distance Feet", "Distance Meters", "Distance Miles", "Distance Kilometers"
// hero6
, "Hero6 Black:TICK[Time]", "Hero6 Black:TSMP[Time]", "Hero6 Black:TMPC[Time]", "Hero6 Black:TMPC[0]", "Hero6 Black:SIUN[Time]"
, "Hero6 Black:ACCL[Time]", "Hero6 Black:GYRO[Time]", "Hero6 Black:GPSF[Time]", "Hero6 Black:GPS5[Time]"
, "Hero6 Black:FCNM[Time]", "Hero6 Black:ISOE[Time]","Hero6 Black:ISOE[0]", "Hero6 Black:SHUT[Time]", "Hero6 Black:WBAL[Time]", "Hero6 Black:WRGB[Time]"
// hero7
, "Hero7 Black:TSMP[Time]", "Hero7 Black:TSMP[0]"
, "Hero7 Black:SIUN[Time]", "Hero7 Black:MTRX[Time]", "Hero7 Black:ORIN[Time]", "Hero7 Black:ORIO[Time]", "Hero7 Black:ACCL[Time]"
, "Hero7 Black:GYRO[Time]", "Hero7 Black:GPSF[Time]", "Hero7 Black:GPSU[Time]", "Hero7 Black:GPSP[Time]", "Hero7 Black:GPS5[Time]", "Hero7 Black:ISOE[Time]"
, "Hero7 Black:SHUT[Time]", "Hero7 Black:ALLD[Time]", "Hero7 Black:WBAL[Time]", "Hero7 Black:WRGB[Time]", "Hero7 Black:UNIF[Time]", "Hero7 Black:SROT[Time]"
, "Hero7 Black:TIMO[Time]"
Tourist
Posts: 8

Re: Combine Several video captures and retain GPS data

[ New ]
That's great information, thanks for sharing! For reference, the Hero 7 Silver has similar columns (maybe not all of them?), but as you might expect, they're prefixed with "Hero 7 Silver" not "Hero 7 Black".
Tourist
Posts: 4

Re: Combine Several video captures and retain GPS data

[ New ]

Anyone know if this is something on GoPro's radar to include in their desktop software or app (or a plug in for a third party)?  The simplest place to add logic to  maintain the metadata in clips is in the editing software (as opposed to users like us exporting/merting csv data).  Ideally they would add this to the desktop software and app, or possibly provide an Adobe Premier plugin?

 

Tourist
Posts: 7

Re: Combine Several video captures and retain GPS data

[ New ]

I first came across all these problems (split files, telemetry, idiot file naming) with the 6, and saw they were all still there in the 7 so I'm not holding my breath. We can hope though!

Tourist
Posts: 5

Re: Combine Several video captures and retain GPS data

[ New ]

I also was looking for a way to combine the GPS data from multiple mp4 files of a gopro fusion camera.

On the SD card of de front facing camera the mp4 files contain the gps data.

The free version of racerender3 can split this data from the mp4 file, but cannot merge multiple files. Probably the paid version can.

I only have 3 files that i want to merge and that gives me 3 dat files in racerender3.

The next command on macbook wil merge these data files.

# cat file1.dat file2.dat file3.dat > combinefile.dat

load the combinefile.dat file in racerender3 without moviefiles and compile the project. next step is edit in adobe premiere or imovie and combine the movie files. It's not the best result, but for me it's good enough.

 

 

Tourist
Posts: 5

Re: Combine Several video captures and retain GPS data

[ New ]
This eventually didn't work for me, because the render time of a movie in the free version has a limit of 3 minutes. After doing some more research I ended up buying racerender3. It's not that expensive and saves me a lot of time. Al lot of people were also very positive about this app. If you want to see the result: https://youtu.be/CL210R53rSg
Hiker
Posts: 6

Re: Combine Several video captures and retain GPS data

[ New ]

Hi everyone, first message!

got myself a nice, fancy go pro 7 hero black ....was looking forward to the GPS feature, went cycling for 3 hours and ended up with a big mess now.

 

so i need to merge the data, the file 7 (for example) wont say 0km and will continue from file 6, and that the map will also merge into a nice loop .

 

can race rendered do that?

 

Also is there an option not to split the files?my YI camera have that option

Tourist
Posts: 7

Re: Combine Several video captures and retain GPS data

[ New ]

There is no option to not split the file, that's just how they do things.

 

File wise, the names are a mess. There is no option to change this either. I ended up downloading a batch rename tool that could apply a regex to wrestle the files into some sort of useful order so that related files would be grouped together.

 

The rename tool I used was this:

https://www.advancedrenamer.com/

 

The regex I used to rename the files was this

find: (\D\D)(\d\d)(\d+)

replace: \1\3_\2

 

It really shouldn't be this hard. But it is, so here we are. 

 

 

Gopro rename options.PNG
Tourist
Posts: 8

Re: Combine Several video captures and retain GPS data

[ New ]

I'd be interested in getting some confirmation that Racerender does handle combining the file correctly, as in handles the accumulation data (and carry over between files) as well as the instance data.

 

This system is an absolute mess. I can't beleive GoPro have left this fault in their product for so long.

 

I don't really use my GoPro any more because of this. The feature I want does not work. I've been holding out for GoPro to release a fix for this, but it appears they aren't interested in providing customer support.

Sightseer
Posts: 2

Re: Combine Several video captures and retain GPS data

[ New ]

I've been using RaceRender for months now. It works great stitching the files together and merging the gps data.

 

The GoPro software remains useless.

Tourist
Posts: 14

Re: Combine Several video captures and retain GPS data

[ New ]

Have you ever wished you'd looked at forums before buying a product, well that's how I feel.

 

I've been using other brands of cameras for recording my motorcycle rides but just "upgraded" to a GoPro HERO 7. I soon discovered their Quik software is total garbage but thought the solution was DashWare. It combines multiple videos but gives the error that data merging.

 

I tried using marce4453's utlitiy to join the data but it wouldn't work either, so today I decided to write my own application. It's very small (230kB) but requires dot net 4.0 or higher (most computers have this already). It's really simple, just click to load the multiple csv files and then click save, the resulting compacted file can be loaded into DashWare along with the merged video and you'll have the full GPS map and associated speed, altitude data etc.

 

If anyone else is interested in it I will post the executable to cnet.download.com later. It's called (very creatively) GoProDataMerge or GPDM. It may work with other cameras, it isn't too specific on the header names but without files to test I can't confirm.

Tourist
Posts: 5

Re: Combine Several video captures and retain GPS data

[ New ]

@zaxs please do share on a WeTransfer/Dropbox/cnet download! Would be very useful.

I have the same problem: merging 2 or 3 files together results in 9 out of 10 times that the GPS data is 'currupt' for some seconds in these few minutes. Even with the CSV merger of marce4453 I have the same problem in Dashware that the telemetry is not recognized (not sure why exactly). 

Maybe your GoProDataMerge application will also be a solution to the issues I run into.

Can you maybe also elaborate on what your merge utility does differently?

Tourist
Posts: 14

Re: Combine Several video captures and retain GPS data

[ New ]

@xfinity it should be live on download.com this week, I will post a link once they confirm it's approved. If you have any problems just send me the CSV files and I will modify the program to work.

Tourist
Posts: 14

Re: Combine Several video captures and retain GPS data

[ New ]

Here's the link on download.com: https://download.cnet.com/GoPro-Data-Merge/3000-20422_4-78084892.html

 

I am sure it will need some tweaks for different files and I am happy to make those changes and release an update.

Tourist
Posts: 1

Re: Combine Several video captures and retain GPS data

[ New ]
You are a god. Thank you so much.
Tourist
Posts: 4

Re: Combine Several video captures and retain GPS data

[ New ]

Hi there

 

Firstly well done for taking this into your own hands - the support doesn't look interested in what is an obvious user need, but I'll leave that alone

 

I've got a 5 black, so your app isn't tested - I wanted to ask if it was possible to manually combine and create a compositve CSV using excel - the data structure looks pretty simple, but you seemed to suggest in your first post that it wasn't as easy as that...

 

Thanks in advance

 

Sam

Tourist
Posts: 4

Re: Combine Several video captures and retain GPS data

[ New ]

Actually, scratch that - your code worked perfectly with my 5 black (Win 10) - thank you so much!!

 

Sam

Tourist
Posts: 14

Re: Combine Several video captures and retain GPS data

[ New ]

Thanks for the feedback, that's great that it works with other models as well.

 

I tried to make it smart on the columns so it doesn't require specific names, but wasn't sure how different the outputs would be.

Tourist
Posts: 4

Re: Combine Several video captures and retain GPS data

[ New ]

So I have been happy rattling through my racing footage, and suddenly dashware starts failing to merge the video files - not the CSVs, it just refuses merge (see attachment)

 

Appreciate this is nothing to do with your app/fix, just wondered if you have had this happen

 

Frustrating, as this was so much easier than fiddling with quik pro

 

Sam

Dashware.PNG
Tourist
Posts: 14

Re: Combine Several video captures and retain GPS data

[ New ]

Is E: a removable drive? If so try moving the files to a different drive.

 

 

Tourist
Posts: 4

Re: Combine Several video captures and retain GPS data

[ New ]

It wasn't but your instincts are right. Embrassingly, I had run out of space on that disk - now humming along happily again :)

 

thanks for responding

 

Sam

Sightseer
Posts: 1

Re: Combine Several video captures and retain GPS data

[ New ]

I tried to merge my csv files generated from DashWare, but the merged file only contains time data, nothing else. Any ideas?

defger.jpg
Tourist
Posts: 14

Re: Combine Several video captures and retain GPS data

[ New ]

Send me your CSV files and I will take a look. PM sent with contact information.

Tourist
Posts: 5

Re: Combine Several video captures and retain GPS data

[ New ]

After a deeper look I think I found the reason for the issues with the data merging tools currently available. I tried using both the "GoProDataMerge" and "GoProCsvMerger" tool by @marce4453 found earlier in this thread here: https://community.gopro.com/t5/Cameras/Combine-Several-video-captures-and-retain-GPS-data/m-p/287975...

 

1) A problem with decimal points in the data, using different language/localization/number formatting settings in the Operating System for users outside the USA (e.g. I am using nl-NL instead of en-US) --> the first file will go fine, but as soon as the second file gets merged and calculations (cumulation of last GlobalTime on top of current file GlobalTime) then the decimals get removed while parsing the string. In the source code when using e.g. C# the decimal.Parse() method would need a second parameter specifying e.g. new CultureInfo("en-US").NumberFormat to explicitly use "." as decimal separator instead of e.g. "," when parsing, because Dashware's files are using format "x.000000" instead of e.g. "x,0000000" (period instead of comma).

 

For example, the GlobalTime in the first file will be exported as "530.950000", while the next line (coming from the second file) will be formatted as "530950000" (note the missing decimal separator).

 

2) The other issue is that different files in the same set (e.g. GH010046_out.csv and GH020046_out.csv) can have different sortings/orders of the header columns. So the merging tools currently have no sorting logic, meaning that values can get mixed up randomly. For example, not the last colum of both files in the same recording session/data set ("Hero7 Black:TIMO[0]" vs "Hero7 Black:SROT[0]", while both files contain the same columns in the end but in a different order): 

 

Headers in example file part 1 "GH010046_out.csv":

GlobalTime,Hero7 Black:TSMP[Time],Hero7 Black:TSMP[0],Hero7 Black:SIUN[Time],Hero7 Black:SIUN[0],Hero7 Black:SIUN[1],Hero7 Black:SIUN[2],Hero7 Black:SIUN[3],Hero7 Black:MTRX[Time],Hero7 Black:MTRX[0],Hero7 Black:MTRX[1],Hero7 Black:MTRX[2],Hero7 Black:MTRX[3],Hero7 Black:MTRX[4],Hero7 Black:MTRX[5],Hero7 Black:MTRX[6],Hero7 Black:MTRX[7],Hero7 Black:MTRX[8],Hero7 Black:ORIN[Time],Hero7 Black:ORIN[0],Hero7 Black:ORIO[Time],Hero7 Black:ORIO[0],Hero7 Black:ACCL[Time],Hero7 Black:ACCL[0],Hero7 Black:ACCL[1],Hero7 Black:ACCL[2],Hero7 Black:GYRO[Time],Hero7 Black:GYRO[0],Hero7 Black:GYRO[1],Hero7 Black:GYRO[2],Hero7 Black:GPSF[Time],Hero7 Black:GPSF[0],Hero7 Black:GPSU[Time],Hero7 Black:GPSU[0],Hero7 Black:GPSP[Time],Hero7 Black:GPSP[0],Hero7 Black:GPS5[Time],Hero7 Black:GPS5[0] (deg),Hero7 Black:GPS5[1] (deg),Hero7 Black:GPS5[2] (m),Hero7 Black:GPS5[3] (m/s),Hero7 Black:GPS5[4] (m/s),Hero7 Black:ISOE[Time],Hero7 Black:ISOE[0],Hero7 Black:SHUT[Time],Hero7 Black:SHUT[0],Hero7 Black:WBAL[Time],Hero7 Black:WBAL[0],Hero7 Black:WRGB[Time],Hero7 Black:WRGB[0],Hero7 Black:WRGB[1],Hero7 Black:WRGB[2],Hero7 Black:YAVG[Time],Hero7 Black:YAVG[0],Hero7 Black:UNIF[Time],Hero7 Black:UNIF[0],Hero7 Black:SCEN[Time],Hero7 Black:SCEN[0],Hero7 Black:SCEN[1],Hero7 Black:HUES[Time],Hero7 Black:HUES[0],Hero7 Black:HUES[1],Hero7 Black:SROT[Time],Hero7 Black:SROT[0],Hero7 Black:TIMO[Time],Hero7 Black:TIMO[0]

 

Headers in example file part 2 "GH020046_out.csv":

GlobalTime,Hero7 Black:TSMP[Time],Hero7 Black:TSMP[0],Hero7 Black:SIUN[Time],Hero7 Black:SIUN[0],Hero7 Black:SIUN[1],Hero7 Black:SIUN[2],Hero7 Black:SIUN[3],Hero7 Black:MTRX[Time],Hero7 Black:MTRX[0],Hero7 Black:MTRX[1],Hero7 Black:MTRX[2],Hero7 Black:MTRX[3],Hero7 Black:MTRX[4],Hero7 Black:MTRX[5],Hero7 Black:MTRX[6],Hero7 Black:MTRX[7],Hero7 Black:MTRX[8],Hero7 Black:ORIN[Time],Hero7 Black:ORIN[0],Hero7 Black:ORIO[Time],Hero7 Black:ORIO[0],Hero7 Black:ACCL[Time],Hero7 Black:ACCL[0],Hero7 Black:ACCL[1],Hero7 Black:ACCL[2],Hero7 Black:GYRO[Time],Hero7 Black:GYRO[0],Hero7 Black:GYRO[1],Hero7 Black:GYRO[2],Hero7 Black:GPSF[Time],Hero7 Black:GPSF[0],Hero7 Black:GPSU[Time],Hero7 Black:GPSU[0],Hero7 Black:GPSP[Time],Hero7 Black:GPSP[0],Hero7 Black:GPS5[Time],Hero7 Black:GPS5[0] (deg),Hero7 Black:GPS5[1] (deg),Hero7 Black:GPS5[2] (m),Hero7 Black:GPS5[3] (m/s),Hero7 Black:GPS5[4] (m/s),Hero7 Black:TIMO[Time],Hero7 Black:TIMO[0],Hero7 Black:ISOE[Time],Hero7 Black:ISOE[0],Hero7 Black:SHUT[Time],Hero7 Black:SHUT[0],Hero7 Black:WBAL[Time],Hero7 Black:WBAL[0],Hero7 Black:WRGB[Time],Hero7 Black:WRGB[0],Hero7 Black:WRGB[1],Hero7 Black:WRGB[2],Hero7 Black:YAVG[Time],Hero7 Black:YAVG[0],Hero7 Black:UNIF[Time],Hero7 Black:UNIF[0],Hero7 Black:SCEN[Time],Hero7 Black:SCEN[0],Hero7 Black:SCEN[1],Hero7 Black:HUES[Time],Hero7 Black:HUES[0],Hero7 Black:HUES[1],Hero7 Black:SROT[Time],Hero7 Black:SROT[0]

 

Not sure why GoPro of Dashware would export different orders of the CSV columns, but I see this happening with my GoPro Hero 7 Black videos.

Tourist
Posts: 14

Re: Combine Several video captures and retain GPS data

[ New ]

That's a simple fix, I will update the app now and submit (they take a few days to approve and load to the site).

Tourist
Posts: 7

Re: Combine Several video captures and retain GPS data

[ New ]

Thanks @xfinity that's good analysis. I can only speak for my code but I was being lazy and not considering localisation at all. 

 

The column ordering one is interesting. Obviously the thing to do there is a full parse of the file to get the columns correct instead of just blindly appending the data in the order it came in. Shows how "lazy" comes back to bite you! (again, talking about myself here.). Although that does mean a reliance on the column names, which in turn requires it to understand the output of each specific camera rather than being generic. Annoying.

Tourist
Posts: 5

Re: Combine Several video captures and retain GPS data

[ New ]

For Dashware and other applications the order of the columns doesnt matter, so a quick and dirty solution could be to sort the record/row based on the key name (assuming each video in the recording session/set does include the same columns, which seems to be the case in all my videos)

 

when using CsvHelper this could be a quick-and-dirty fix (still being lazy :)) without requiring looping through all files first:

 

var sortedLastRow = ((IDictionary<string, object>)lastRow)
.Select(x => new { x.Key, x.Value })
.OrderBy(x => x.Key);

if (headersWritten == false)
{
sortedLastRow.ToList().ForEach(x =>
{
csvWriter.WriteField(x.Key);
});
csvWriter.NextRecord();
headersWritten = true;
}

sortedLastRow.ToList().ForEach(x =>
{
csvWriter.WriteField(x.Value);
});


//csvWriter.WriteRecord(lastRow); //Disabled this, added above logic to sort the columns
csvWriter.NextRecord();

Tourist
Posts: 14

Re: Combine Several video captures and retain GPS data

[ New ]

Column sequence isn't important with my app and it doesn't require exact names (although it could be improved).

 

I've fixed for localization and uploaded to "download.com" so it should be available later this week.

Tourist
Posts: 14

Re: Combine Several video captures and retain GPS data

[ New ]

The update to fix localization is released: https://download.cnet.com/GoPro-Data-Merge/3000-20422_4-78084892.html

 

If anyone is interested there's a "secret" mode in the app that can display the full grid which allows editing or appending files etc.

 

The GPS fix feature isn't completely solved yet, what I want to do is remove any data that doesn't appear to be accurate such as at start up before it locks on to a clean signal (I see it jumping around which messes up my map plot) or if you go in a tunnel and it goes crazy. I'm comparing the coordinates from previous points and determining if the change is plausible or not. If not I replace the values with the previous points so the map stays static until data becomes stable again.

gpdm_adv.jpg
Tourist
Posts: 5

Re: Combine Several video captures and retain GPS data

[ New ]

@jakobj27034 wrote:

I tried to merge my csv files generated from DashWare, but the merged file only contains time data, nothing else. Any ideas?


@jakobj27034 I had the same issue before, still caused by an incorrect Data Profile in Dashware.

 

When Dashware is configured correctly with the right "Hero7BlackGPMF.xml" profile then the columns of the exported CSV should look something like this (first row in the CSV file):

GlobalTime,Hero7 Black:TSMP[Time],Hero7 Black:TSMP[0],Hero7 Black:SIUN[Time],Hero7 Black:SIUN[0],Hero7 Black:SIUN[1],Hero7 Black:SIUN[2],Hero7 Black:SIUN[3],Hero7 Black:MTRX[Time],Hero7 Black:MTRX[0],Hero7 Black:MTRX[1],Hero7 Black:MTRX[2],Hero7 Black:MTRX[3],Hero7 Black:MTRX[4],Hero7 Black:MTRX[5],Hero7 Black:MTRX[6],Hero7 Black:MTRX[7],Hero7 Black:MTRX[8],Hero7 Black:ORIN[Time],Hero7 Black:ORIN[0],Hero7 Black:ORIO[Time],Hero7 Black:ORIO[0],Hero7 Black:ACCL[Time],Hero7 Black:ACCL[0],Hero7 Black:ACCL[1],Hero7 Black:ACCL[2],Hero7 Black:GYRO[Time],Hero7 Black:GYRO[0],Hero7 Black:GYRO[1],Hero7 Black:GYRO[2],Hero7 Black:GPSF[Time],Hero7 Black:GPSF[0],Hero7 Black:GPSU[Time],Hero7 Black:GPSU[0],DateTime,Hero7 Black:GPSP[Time],Hero7 Black:GPSP[0],Precision_i,Hero7 Black:GPS5[Time],Hero7 Black:GPS5[0] (deg),Hero7 Black:GPS5[1] (deg),Hero7 Black:GPS5[2] (m),Hero7 Black:GPS5[3] (m/s),Hero7 Black:GPS5[4] (m/s),Latitude_i,Longitude_i,Elevation_i,Speed_i,Elevation Meters,Elevation Feet,Total Ascent Meters,Total Descent Meters,Altitude Min Meters,Altitude Max Meters,Total Ascent Feet,Total Descent Feet,Altitude Min Feet,Altitude Max Feet,Speed KPH,Speed MPH,GPS_X,GPS_Y,Distance Feet,Distance Meters,Distance Miles,Distance Kilometers,Raw Heading,Heading,Raw GPS AccelGs,Raw GPS LatGs,GPS AccelGs,GPS LatGs,GPS 2D Gs,Hero7 Black:ISOE[Time],Hero7 Black:ISOE[0],Hero7 Black:SHUT[Time],Hero7 Black:SHUT[0],Hero7 Black:WBAL[Time],Hero7 Black:WBAL[0],Hero7 Black:WRGB[Time],Hero7 Black:WRGB[0],Hero7 Black:WRGB[1],Hero7 Black:WRGB[2],Hero7 Black:YAVG[Time],Hero7 Black:YAVG[0],Hero7 Black:UNIF[Time],Hero7 Black:UNIF[0],Hero7 Black:SCEN[Time],Hero7 Black:SCEN[0],Hero7 Black:SCEN[1],Hero7 Black:HUES[Time],Hero7 Black:HUES[0],Hero7 Black:HUES[1],Hero7 Black:SROT[Time],Hero7 Black:SROT[0],Hero7 Black:TIMO[Time],Hero7 Black:TIMO[0]

 

However, if the "Hero7BlackGPMF.xml" profile is not configured correctly then an export is created, but it is missing the crucial standardized & calculated columns. The first row containing the columns looks like this then:

GlobalTime,Hero7 Black:TSMP[Time],Hero7 Black:TSMP[0],Hero7 Black:SIUN[Time],Hero7 Black:SIUN[0],Hero7 Black:SIUN[1],Hero7 Black:SIUN[2],Hero7 Black:SIUN[3],Hero7 Black:MTRX[Time],Hero7 Black:MTRX[0],Hero7 Black:MTRX[1],Hero7 Black:MTRX[2],Hero7 Black:MTRX[3],Hero7 Black:MTRX[4],Hero7 Black:MTRX[5],Hero7 Black:MTRX[6],Hero7 Black:MTRX[7],Hero7 Black:MTRX[8],Hero7 Black:ORIN[Time],Hero7 Black:ORIN[0],Hero7 Black:ORIO[Time],Hero7 Black:ORIO[0],Hero7 Black:ACCL[Time],Hero7 Black:ACCL[0],Hero7 Black:ACCL[1],Hero7 Black:ACCL[2],Hero7 Black:GYRO[Time],Hero7 Black:GYRO[0],Hero7 Black:GYRO[1],Hero7 Black:GYRO[2],Hero7 Black:GPSF[Time],Hero7 Black:GPSF[0],Hero7 Black:GPSU[Time],Hero7 Black:GPSU[0],Hero7 Black:GPSP[Time],Hero7 Black:GPSP[0],Hero7 Black:GPS5[Time],Hero7 Black:GPS5[0] (deg),Hero7 Black:GPS5[1] (deg),Hero7 Black:GPS5[2] (m),Hero7 Black:GPS5[3] (m/s),Hero7 Black:GPS5[4] (m/s),Hero7 Black:ISOE[Time],Hero7 Black:ISOE[0],Hero7 Black:SHUT[Time],Hero7 Black:SHUT[0],Hero7 Black:WBAL[Time],Hero7 Black:WBAL[0],Hero7 Black:WRGB[Time],Hero7 Black:WRGB[0],Hero7 Black:WRGB[1],Hero7 Black:WRGB[2],Hero7 Black:YAVG[Time],Hero7 Black:YAVG[0],Hero7 Black:UNIF[Time],Hero7 Black:UNIF[0],Hero7 Black:SCEN[Time],Hero7 Black:SCEN[0],Hero7 Black:SCEN[1],Hero7 Black:HUES[Time],Hero7 Black:HUES[0],Hero7 Black:HUES[1],Hero7 Black:SROT[Time],Hero7 Black:SROT[0],Hero7 Black:TIMO[Time],Hero7 Black:TIMO[0]

 

Notice that the following columns are missing, and which are expected by the GPDM tool (all other columns are deleted/not copied by this specific merge tool):

Latitude_i,Longitude_i,Elevation_i,Speed_i,Elevation Meters,Elevation Feet,Total Ascent Meters,Total Descent Meters,Altitude Min Meters,Altitude Max Meters,Total Ascent Feet,Total Descent Feet,Altitude Min Feet,Altitude Max Feet,Speed KPH,Speed MPH,GPS_X,GPS_Y,Distance Feet,Distance Meters,Distance Miles,Distance Kilometers,Raw Heading,Heading,Raw GPS AccelGs,Raw GPS LatGs,GPS AccelGs,GPS LatGs,GPS 2D Gs

 

The "GoPro CSV Merger" tool does not have this issue (it still merges all existing columns and doesn't remove columns), but the most important columns are still missing, which are used by Dashware, resulting in no visible data in the gauges (or constant 0 values).

 

What I did to fix this issue (using the GoPro Hero 7 Black running the latest firmware v1.70/23 jan 2019, and the latest Dashware version 1.9.1):

  1. Uninstall Dashware completely.
  2. Make sure to delete the following directory if it is still available after uninstall: "C:\Program Files\DashWare\"
  3. Make sure to delete the following directory if it is still available after uninstall: "C:\Users\%username%\Documents\DashWare"
  4. Download the latest version of Dashware (currently 1.9.1) http://www.dashware.net/dashware-download/
  5. If you have GoPro Quik installed, then copy the "Hero7BlackGPMF.xml" file located at "C:\Program Files\GoPro\GoPro Desktop App\telemetrydata\DataProfiles\Hero7BlackGPMF.xml". If you don't have it installed, I found a copy online via this link that worked for me: https://www64.zippyshare.com/v/dheBhIMV/file.html
  6. Paste the "Hero7BlackGPMF.xml" file into the following Dashware directory: "C:\Program Files\DashWare\DataTool\DataProfiles"
  7. Run Dashware.
  8. In the menu go to "File" > "Quick Project Creator" and select your *.mp4 video file (uncheck Copy file to project directory).
  9. At the main screen on the right side under "Input Settings", click the "+" sign next to "Video:" and select your *.mp4 video file.
  10. Next, below this, click the "+" sign next to the "Data File(s):" and select the same *.mp4 video file as you selected above. At "Choose a data profile:" select "GoPro" and click "Add".
  11. In your Dashware project directory ("C:\Users\%username%\Documents\DashWare Projects\" with your project's subdirectory) a CSV file is generated for the video file, which now should contain the missing columns. You can open the CSV file in e.g. Notepad or Excel, and you can search for the text "Latitude_i" to see if column is available now, or is still missing.

 

Hope this helps!