Using Dropbox to Sync File GeoDatabases? The Next CEO of Stack OverflowFile GDB Feature Classes Converting Themselves to File GDB TablesImplementing version control system for geospatial data?Reading file geodatabases without using ArcObjects or any other licensesQGIS Merged Layers/ Cannot Find Windows “minidump” to Restore Past version off DropboxHost LeafLet Page on DropboxMoving file geodatabases in python without ArcPyHow to sync/update an out-of-sync replica?Handling “bad layers” with Esri File Geodatabases?MapServer 7 and File Geodatabases and ExpressionsUsing arcpy.da.Walk to go through many file geodatabasesMerging geodatabases using ArcPy?Creating file geodatabases using list as filenames in ArcPy?
Why is there a PLL in CPU?
Why doesn't a table tennis ball float on the surface? How do we calculate buoyancy here?
Is it okay to store user locations?
Opposite of a diet
What is the purpose of the Evocation wizard's Potent Cantrip feature?
The King's new dress
Return the Closest Prime Number
Anatomically Correct Strange Women In Ponds Distributing Swords
How do I get the green key off the shelf in the Dobby level of Lego Harry Potter 2?
Customer Requests (Sometimes) Drive Me Bonkers!
How can I get through very long and very dry, but also very useful technical documents when learning a new tool?
When did Lisp start using symbols for arithmetic?
Rotate a column
Too much space between section and text in a twocolumn document
Would this house-rule that treats advantage as a +1 to the roll instead (and disadvantage as -1) and allows them to stack be balanced?
How to make a software documentation "officially" citable?
Was a professor correct to chastise me for writing "Prof. X" rather than "Professor X"?
Are there languages with no euphemisms?
How to safely derail a train during transit?
How do I go from 300 unfinished/half written blog posts, to published posts?
Why does GHC infer a monomorphic type here, even with MonomorphismRestriction disabled?
When airplanes disconnect from a tanker during air to air refueling, why do they bank so sharply to the right?
How do scammers retract money, while you can’t?
If I blow insulation everywhere in my attic except the door trap, will heat escape through it?
Using Dropbox to Sync File GeoDatabases?
The Next CEO of Stack OverflowFile GDB Feature Classes Converting Themselves to File GDB TablesImplementing version control system for geospatial data?Reading file geodatabases without using ArcObjects or any other licensesQGIS Merged Layers/ Cannot Find Windows “minidump” to Restore Past version off DropboxHost LeafLet Page on DropboxMoving file geodatabases in python without ArcPyHow to sync/update an out-of-sync replica?Handling “bad layers” with Esri File Geodatabases?MapServer 7 and File Geodatabases and ExpressionsUsing arcpy.da.Walk to go through many file geodatabasesMerging geodatabases using ArcPy?Creating file geodatabases using list as filenames in ArcPy?
I've been able to accumulate a large free dropbox account and use it to Sync my GIS databases across several different PC's. I've just started doing this and havent had any problems thus far. But I am getting some red flags of warning that are scaring me.
There seems to be several duplicate files in the GeoDatabases. Usually something like:
a00000017.gdbtable (GIS-HP duplicate Copy)
I can manually delete these files with no problem, but I have no idea why they are popping up.
Also, instead of removing file locks, Dropbox keeps saying it is renaming them. So when I release one lock and gain another, it renames the file instead of deleting it and creating a new one. Although it still does delete the lock file when I have fully disconnected from the GeoDatabase.
Like I said, I haven't had any issues so far, but I am worried about long term data corruption while using Dropbox.
Has anyone used Dropbox and found problems using it to sync GIS data?
file-geodatabase replication dropbox
edited yesterday
PolyGeo♦
53.8k1781245
asked Aug 21 '13 at 15:12
Cody BrownCody Brown
2,44033366
add a comment |
I've been able to accumulate a large free dropbox account and use it to Sync my GIS databases across several different PC's. I've just started doing this and havent had any problems thus far. But I am getting some red flags of warning that are scaring me.
There seems to be several duplicate files in the GeoDatabases. Usually something like:
a00000017.gdbtable (GIS-HP duplicate Copy)
I can manually delete these files with no problem, but I have no idea why they are popping up.
Also, instead of removing file locks, Dropbox keeps saying it is renaming them. So when I release one lock and gain another, it renames the file instead of deleting it and creating a new one. Although it still does delete the lock file when I have fully disconnected from the GeoDatabase.
Like I said, I haven't had any issues so far, but I am worried about long term data corruption while using Dropbox.
Has anyone used Dropbox and found problems using it to sync GIS data?
file-geodatabase replication dropbox
edited yesterday
PolyGeo♦
53.8k1781245
asked Aug 21 '13 at 15:12
Cody BrownCody Brown
2,44033366
3
I don't have a definitive answer, but I'd suggest that you're wise to be cautious. I don't believe that DropBox is intended for accessing a complex data format like a file geodatabase, which consists of multiple discrete files, on multiple PCs simultaneously. This post shows some potential issues to be aware of
– Stephen Lead
Aug 21 '13 at 15:18
@StephenLead I've been backing up the Dropbox data daily. I used Amazon Cloud Drive for about a year with no issues, but that is so slow for syncing. Dropbox works way quicker and simpler. I really hope that it can handle it.
– Cody Brown
Aug 21 '13 at 15:28
2
It is better to compress the file geodatabase before it is used to share resources.arcgis.com/en/help/main/10.2/index.html#//… it is not designed to be edited with more than one user at one time - lck files should prevent this.
– Mapperz♦
Aug 21 '13 at 16:31
Solid idea @Mapperz
– Cody Brown
Aug 21 '13 at 16:42
add a comment |
I've been able to accumulate a large free dropbox account and use it to Sync my GIS databases across several different PC's. I've just started doing this and havent had any problems thus far. But I am getting some red flags of warning that are scaring me.
There seems to be several duplicate files in the GeoDatabases. Usually something like:
a00000017.gdbtable (GIS-HP duplicate Copy)
I can manually delete these files with no problem, but I have no idea why they are popping up.
Also, instead of removing file locks, Dropbox keeps saying it is renaming them. So when I release one lock and gain another, it renames the file instead of deleting it and creating a new one. Although it still does delete the lock file when I have fully disconnected from the GeoDatabase.
Like I said, I haven't had any issues so far, but I am worried about long term data corruption while using Dropbox.
Has anyone used Dropbox and found problems using it to sync GIS data?
file-geodatabase replication dropbox
edited yesterday
PolyGeo♦
53.8k1781245
asked Aug 21 '13 at 15:12
Cody BrownCody Brown
2,44033366
I've been able to accumulate a large free dropbox account and use it to Sync my GIS databases across several different PC's. I've just started doing this and havent had any problems thus far. But I am getting some red flags of warning that are scaring me.
There seems to be several duplicate files in the GeoDatabases. Usually something like:
a00000017.gdbtable (GIS-HP duplicate Copy)
I can manually delete these files with no problem, but I have no idea why they are popping up.
Also, instead of removing file locks, Dropbox keeps saying it is renaming them. So when I release one lock and gain another, it renames the file instead of deleting it and creating a new one. Although it still does delete the lock file when I have fully disconnected from the GeoDatabase.
Like I said, I haven't had any issues so far, but I am worried about long term data corruption while using Dropbox.
Has anyone used Dropbox and found problems using it to sync GIS data?
file-geodatabase replication dropbox
file-geodatabase replication dropbox
edited yesterday
PolyGeo♦
53.8k1781245
asked Aug 21 '13 at 15:12
Cody BrownCody Brown
2,44033366
edited yesterday
PolyGeo♦
53.8k1781245
asked Aug 21 '13 at 15:12
Cody BrownCody Brown
2,44033366
edited yesterday
PolyGeo♦
53.8k1781245
edited yesterday
PolyGeo♦
53.8k1781245
edited yesterday
PolyGeo♦
53.8k1781245
53.8k1781245
asked Aug 21 '13 at 15:12
Cody BrownCody Brown
2,44033366
asked Aug 21 '13 at 15:12
Cody BrownCody Brown
2,44033366
asked Aug 21 '13 at 15:12
Cody BrownCody Brown
2,44033366
2,44033366
3
I don't have a definitive answer, but I'd suggest that you're wise to be cautious. I don't believe that DropBox is intended for accessing a complex data format like a file geodatabase, which consists of multiple discrete files, on multiple PCs simultaneously. This post shows some potential issues to be aware of
– Stephen Lead
Aug 21 '13 at 15:18
@StephenLead I've been backing up the Dropbox data daily. I used Amazon Cloud Drive for about a year with no issues, but that is so slow for syncing. Dropbox works way quicker and simpler. I really hope that it can handle it.
– Cody Brown
Aug 21 '13 at 15:28
2
It is better to compress the file geodatabase before it is used to share resources.arcgis.com/en/help/main/10.2/index.html#//… it is not designed to be edited with more than one user at one time - lck files should prevent this.
– Mapperz♦
Aug 21 '13 at 16:31
Solid idea @Mapperz
– Cody Brown
Aug 21 '13 at 16:42
add a comment |
3
I don't have a definitive answer, but I'd suggest that you're wise to be cautious. I don't believe that DropBox is intended for accessing a complex data format like a file geodatabase, which consists of multiple discrete files, on multiple PCs simultaneously. This post shows some potential issues to be aware of
– Stephen Lead
Aug 21 '13 at 15:18
@StephenLead I've been backing up the Dropbox data daily. I used Amazon Cloud Drive for about a year with no issues, but that is so slow for syncing. Dropbox works way quicker and simpler. I really hope that it can handle it.
– Cody Brown
Aug 21 '13 at 15:28
2
It is better to compress the file geodatabase before it is used to share resources.arcgis.com/en/help/main/10.2/index.html#//… it is not designed to be edited with more than one user at one time - lck files should prevent this.
– Mapperz♦
Aug 21 '13 at 16:31
Solid idea @Mapperz
– Cody Brown
Aug 21 '13 at 16:42
3
3
I don't have a definitive answer, but I'd suggest that you're wise to be cautious. I don't believe that DropBox is intended for accessing a complex data format like a file geodatabase, which consists of multiple discrete files, on multiple PCs simultaneously. This post shows some potential issues to be aware of
– Stephen Lead
Aug 21 '13 at 15:18
I don't have a definitive answer, but I'd suggest that you're wise to be cautious. I don't believe that DropBox is intended for accessing a complex data format like a file geodatabase, which consists of multiple discrete files, on multiple PCs simultaneously. This post shows some potential issues to be aware of
– Stephen Lead
Aug 21 '13 at 15:18
@StephenLead I've been backing up the Dropbox data daily. I used Amazon Cloud Drive for about a year with no issues, but that is so slow for syncing. Dropbox works way quicker and simpler. I really hope that it can handle it.
– Cody Brown
Aug 21 '13 at 15:28
@StephenLead I've been backing up the Dropbox data daily. I used Amazon Cloud Drive for about a year with no issues, but that is so slow for syncing. Dropbox works way quicker and simpler. I really hope that it can handle it.
– Cody Brown
Aug 21 '13 at 15:28
2
2
It is better to compress the file geodatabase before it is used to share resources.arcgis.com/en/help/main/10.2/index.html#//… it is not designed to be edited with more than one user at one time - lck files should prevent this.
– Mapperz♦
Aug 21 '13 at 16:31
It is better to compress the file geodatabase before it is used to share resources.arcgis.com/en/help/main/10.2/index.html#//… it is not designed to be edited with more than one user at one time - lck files should prevent this.
– Mapperz♦
Aug 21 '13 at 16:31
Solid idea @Mapperz
– Cody Brown
Aug 21 '13 at 16:42
Solid idea @Mapperz
– Cody Brown
Aug 21 '13 at 16:42
add a comment |
2 Answers
2
active
oldest
votes
ESRI intentionally obfuscated the FileGDB file structure so people wouldn't try and mess with it internally:
A file geodatabase is stored as a folder of files. When you view the
folder with Windows Explorer, apart from the .gdb extension, it looks
like any other folder and you can view its contents. The folder
contains cryptically named files that hold geographic data, attribute
data, index files, .lock files, and a signature file, as well as other
files. Each feature class or table in the geodatabase is stored in two
or more files. By design, it is difficult, if not impossible, to tell
which files make up any given dataset.
ESRI recommends that you copy FileGDB's with their tools and not your OS tools. However, it is possible to copy the whole database or rename it:
Although not recommended, it is possible to copy a file geodatabase to
another location by copying the folder to another location (if you are
using a Windows Vista operating system, you also need to change the
extension of the new folder to .gdb; otherwise, ArcGIS cannot
recognize it as a file geodatabase). Only copy a file geodatabase with
Windows Explorer when you are certain there are no other processes
accessing the data. The problem is that it can be difficult to tell
whether other processes are accessing the data. You might think there
are no other processes; however, a process on your computer or another
one could be accessing it. If you happen to copy the folder when
another process is accessing the data, the data may not copy properly.
An improper copy operation is often not apparent—you might not notice
any problems until you access a certain part of the new copy a few
days later.
I would be wary of using dropbox to sync the files individually. The database structure just isn't built to be manipulated in that way. If you dataset is small enough, it may be a good application for a personal geodatabase or a sqlite database (though I haven't seen it confirmed that ArcGIS can write sqlite) if you have simple geometry.
There is a tool that will recover corrupted geodatabase in the event they are damaged:
answered Aug 21 '13 at 15:28
DPierceDPierce
1,904612
2
You quote: 'you might not notice any problems until you access a certain part of the new copy a few days later' scares the crap out of me. It would probably be way smarter to use shapefiles instead of a GeoDatabase.
– Cody Brown
Aug 21 '13 at 15:33
I have been using SkyDrive to sync file gdbs without issue for almost a year now. When we have proxy issues which sometimes means I can't access Microsoft services, then I normally close my SkyDrive and only allow it to sync again once access has been restored. I have edited toolboxes with models and scripts when my SkyDrive is open but not syncing properly, and I have not lost anything yet.
– Cindy Jayakumar
Aug 23 '13 at 6:08
1
FWIW I was editing a complex polygon layer in a file GDB so I disconnected Dropbox while I was working on it. I reconnected Dropbox after editing, so the files were backed up. 24 hours later, all my edits are gone, and (since I had disconnected dropbox during editing) there are no intermediate files to revert to. So it looks like Dropbox may sometimes corrupt a file geodatabase.
– Stephen Lead
May 14 '14 at 10:28
add a comment |
If you need to have multiple PCs accessing files, it would seem to make more sense to migrate to a true spatial server setup rather than an ad hoc one using Dropbox. This could be either a hosted server running ArcGIS or a PostGIS database running on a cloud web service like AWS. My preference would be for the latter. It'll take a little more work to set up, but you can define your own relations and scale up to as many computers as you need (depending on how you configure AWS), all for much less than you'd be paying Esri.
There are several SO posts that discuss PostGIS and ArcGIS here and here. The second one seems to have more up-to-date information in the comments below the accepted answer.
Without something more robust handling the traffic from the different clients, you're probably going to be constantly dealing with data corruption issues. At the very least, if you moved to shapefiles, you should check out GeoGit. This would give you basic revision control so shapefiles edited on one computer can be merged with edits from another without being completely overwritten.
edited yesterday
PolyGeo♦
53.8k1781245
answered Aug 22 '13 at 19:46
Richard DRichard D
42028
Web servers are a great way to manage GIS Data across several PC's. In fact I agree, it's probably the best way. The issue with it is bandwidth and processing speed. Usually the internet speed offsite is high enough to work with the data. But, if I want to do some heavy analysis of the data, there is no way I want to be pulling it across the web. That's why I prefer a solution like Dropbox. I show up onsite and the GeoDatabases are all up to date and stored locally on the PC. Great way to work.
– Cody Brown
Aug 22 '13 at 20:10
3
@codybrown But you're still dealing with pushing data over the Internet with Dropbox, not to mention the sync issues you're getting, probably because it's taking so long to update from one client to another. At least a server would be optimized for that task. If you're really just looking to have a storage solution for data while you work from home, work, or client site, then a large removal hard drive is the only way to avoid latency and potential sync conflicts, assuming you're the only one using the data at any given time.
– Richard D
Aug 22 '13 at 21:41
add a comment |
Your Answer
StackExchange.ready(function()
var channelOptions =
tags: "".split(" "),
id: "79"
;
initTagRenderer("".split(" "), "".split(" "), channelOptions);
StackExchange.using("externalEditor", function()
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled)
StackExchange.using("snippets", function()
createEditor();
);
else
createEditor();
);
function createEditor()
StackExchange.prepareEditor(
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: false,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: null,
bindNavPrevention: true,
postfix: "",
imageUploader:
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
,
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
);
);
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function ()
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fgis.stackexchange.com%2fquestions%2f69307%2fusing-dropbox-to-sync-file-geodatabases%23new-answer', 'question_page');
);
Post as a guest
Required, but never shown
2 Answers
2
active
oldest
votes
2 Answers
2
active
oldest
votes
active
oldest
votes
active
oldest
votes
ESRI intentionally obfuscated the FileGDB file structure so people wouldn't try and mess with it internally:
A file geodatabase is stored as a folder of files. When you view the
folder with Windows Explorer, apart from the .gdb extension, it looks
like any other folder and you can view its contents. The folder
contains cryptically named files that hold geographic data, attribute
data, index files, .lock files, and a signature file, as well as other
files. Each feature class or table in the geodatabase is stored in two
or more files. By design, it is difficult, if not impossible, to tell
which files make up any given dataset.
ESRI recommends that you copy FileGDB's with their tools and not your OS tools. However, it is possible to copy the whole database or rename it:
Although not recommended, it is possible to copy a file geodatabase to
another location by copying the folder to another location (if you are
using a Windows Vista operating system, you also need to change the
extension of the new folder to .gdb; otherwise, ArcGIS cannot
recognize it as a file geodatabase). Only copy a file geodatabase with
Windows Explorer when you are certain there are no other processes
accessing the data. The problem is that it can be difficult to tell
whether other processes are accessing the data. You might think there
are no other processes; however, a process on your computer or another
one could be accessing it. If you happen to copy the folder when
another process is accessing the data, the data may not copy properly.
An improper copy operation is often not apparent—you might not notice
any problems until you access a certain part of the new copy a few
days later.
I would be wary of using dropbox to sync the files individually. The database structure just isn't built to be manipulated in that way. If you dataset is small enough, it may be a good application for a personal geodatabase or a sqlite database (though I haven't seen it confirmed that ArcGIS can write sqlite) if you have simple geometry.
There is a tool that will recover corrupted geodatabase in the event they are damaged:
answered Aug 21 '13 at 15:28
DPierceDPierce
1,904612
2
You quote: 'you might not notice any problems until you access a certain part of the new copy a few days later' scares the crap out of me. It would probably be way smarter to use shapefiles instead of a GeoDatabase.
– Cody Brown
Aug 21 '13 at 15:33
I have been using SkyDrive to sync file gdbs without issue for almost a year now. When we have proxy issues which sometimes means I can't access Microsoft services, then I normally close my SkyDrive and only allow it to sync again once access has been restored. I have edited toolboxes with models and scripts when my SkyDrive is open but not syncing properly, and I have not lost anything yet.
– Cindy Jayakumar
Aug 23 '13 at 6:08
1
FWIW I was editing a complex polygon layer in a file GDB so I disconnected Dropbox while I was working on it. I reconnected Dropbox after editing, so the files were backed up. 24 hours later, all my edits are gone, and (since I had disconnected dropbox during editing) there are no intermediate files to revert to. So it looks like Dropbox may sometimes corrupt a file geodatabase.
– Stephen Lead
May 14 '14 at 10:28
add a comment |
ESRI intentionally obfuscated the FileGDB file structure so people wouldn't try and mess with it internally:
A file geodatabase is stored as a folder of files. When you view the
folder with Windows Explorer, apart from the .gdb extension, it looks
like any other folder and you can view its contents. The folder
contains cryptically named files that hold geographic data, attribute
data, index files, .lock files, and a signature file, as well as other
files. Each feature class or table in the geodatabase is stored in two
or more files. By design, it is difficult, if not impossible, to tell
which files make up any given dataset.
ESRI recommends that you copy FileGDB's with their tools and not your OS tools. However, it is possible to copy the whole database or rename it:
Although not recommended, it is possible to copy a file geodatabase to
another location by copying the folder to another location (if you are
using a Windows Vista operating system, you also need to change the
extension of the new folder to .gdb; otherwise, ArcGIS cannot
recognize it as a file geodatabase). Only copy a file geodatabase with
Windows Explorer when you are certain there are no other processes
accessing the data. The problem is that it can be difficult to tell
whether other processes are accessing the data. You might think there
are no other processes; however, a process on your computer or another
one could be accessing it. If you happen to copy the folder when
another process is accessing the data, the data may not copy properly.
An improper copy operation is often not apparent—you might not notice
any problems until you access a certain part of the new copy a few
days later.
I would be wary of using dropbox to sync the files individually. The database structure just isn't built to be manipulated in that way. If you dataset is small enough, it may be a good application for a personal geodatabase or a sqlite database (though I haven't seen it confirmed that ArcGIS can write sqlite) if you have simple geometry.
There is a tool that will recover corrupted geodatabase in the event they are damaged:
answered Aug 21 '13 at 15:28
DPierceDPierce
1,904612
2
You quote: 'you might not notice any problems until you access a certain part of the new copy a few days later' scares the crap out of me. It would probably be way smarter to use shapefiles instead of a GeoDatabase.
– Cody Brown
Aug 21 '13 at 15:33
I have been using SkyDrive to sync file gdbs without issue for almost a year now. When we have proxy issues which sometimes means I can't access Microsoft services, then I normally close my SkyDrive and only allow it to sync again once access has been restored. I have edited toolboxes with models and scripts when my SkyDrive is open but not syncing properly, and I have not lost anything yet.
– Cindy Jayakumar
Aug 23 '13 at 6:08
1
FWIW I was editing a complex polygon layer in a file GDB so I disconnected Dropbox while I was working on it. I reconnected Dropbox after editing, so the files were backed up. 24 hours later, all my edits are gone, and (since I had disconnected dropbox during editing) there are no intermediate files to revert to. So it looks like Dropbox may sometimes corrupt a file geodatabase.
– Stephen Lead
May 14 '14 at 10:28
add a comment |
ESRI intentionally obfuscated the FileGDB file structure so people wouldn't try and mess with it internally:
A file geodatabase is stored as a folder of files. When you view the
folder with Windows Explorer, apart from the .gdb extension, it looks
like any other folder and you can view its contents. The folder
contains cryptically named files that hold geographic data, attribute
data, index files, .lock files, and a signature file, as well as other
files. Each feature class or table in the geodatabase is stored in two
or more files. By design, it is difficult, if not impossible, to tell
which files make up any given dataset.
ESRI recommends that you copy FileGDB's with their tools and not your OS tools. However, it is possible to copy the whole database or rename it:
Although not recommended, it is possible to copy a file geodatabase to
another location by copying the folder to another location (if you are
using a Windows Vista operating system, you also need to change the
extension of the new folder to .gdb; otherwise, ArcGIS cannot
recognize it as a file geodatabase). Only copy a file geodatabase with
Windows Explorer when you are certain there are no other processes
accessing the data. The problem is that it can be difficult to tell
whether other processes are accessing the data. You might think there
are no other processes; however, a process on your computer or another
one could be accessing it. If you happen to copy the folder when
another process is accessing the data, the data may not copy properly.
An improper copy operation is often not apparent—you might not notice
any problems until you access a certain part of the new copy a few
days later.
I would be wary of using dropbox to sync the files individually. The database structure just isn't built to be manipulated in that way. If you dataset is small enough, it may be a good application for a personal geodatabase or a sqlite database (though I haven't seen it confirmed that ArcGIS can write sqlite) if you have simple geometry.
There is a tool that will recover corrupted geodatabase in the event they are damaged:
answered Aug 21 '13 at 15:28
DPierceDPierce
1,904612
ESRI intentionally obfuscated the FileGDB file structure so people wouldn't try and mess with it internally:
A file geodatabase is stored as a folder of files. When you view the
folder with Windows Explorer, apart from the .gdb extension, it looks
like any other folder and you can view its contents. The folder
contains cryptically named files that hold geographic data, attribute
data, index files, .lock files, and a signature file, as well as other
files. Each feature class or table in the geodatabase is stored in two
or more files. By design, it is difficult, if not impossible, to tell
which files make up any given dataset.
ESRI recommends that you copy FileGDB's with their tools and not your OS tools. However, it is possible to copy the whole database or rename it:
Although not recommended, it is possible to copy a file geodatabase to
another location by copying the folder to another location (if you are
using a Windows Vista operating system, you also need to change the
extension of the new folder to .gdb; otherwise, ArcGIS cannot
recognize it as a file geodatabase). Only copy a file geodatabase with
Windows Explorer when you are certain there are no other processes
accessing the data. The problem is that it can be difficult to tell
whether other processes are accessing the data. You might think there
are no other processes; however, a process on your computer or another
one could be accessing it. If you happen to copy the folder when
another process is accessing the data, the data may not copy properly.
An improper copy operation is often not apparent—you might not notice
any problems until you access a certain part of the new copy a few
days later.
I would be wary of using dropbox to sync the files individually. The database structure just isn't built to be manipulated in that way. If you dataset is small enough, it may be a good application for a personal geodatabase or a sqlite database (though I haven't seen it confirmed that ArcGIS can write sqlite) if you have simple geometry.
There is a tool that will recover corrupted geodatabase in the event they are damaged:
answered Aug 21 '13 at 15:28
DPierceDPierce
1,904612
answered Aug 21 '13 at 15:28
DPierceDPierce
1,904612
answered Aug 21 '13 at 15:28
DPierceDPierce
1,904612
answered Aug 21 '13 at 15:28
DPierceDPierce
1,904612
1,904612
2
You quote: 'you might not notice any problems until you access a certain part of the new copy a few days later' scares the crap out of me. It would probably be way smarter to use shapefiles instead of a GeoDatabase.
– Cody Brown
Aug 21 '13 at 15:33
I have been using SkyDrive to sync file gdbs without issue for almost a year now. When we have proxy issues which sometimes means I can't access Microsoft services, then I normally close my SkyDrive and only allow it to sync again once access has been restored. I have edited toolboxes with models and scripts when my SkyDrive is open but not syncing properly, and I have not lost anything yet.
– Cindy Jayakumar
Aug 23 '13 at 6:08
1
FWIW I was editing a complex polygon layer in a file GDB so I disconnected Dropbox while I was working on it. I reconnected Dropbox after editing, so the files were backed up. 24 hours later, all my edits are gone, and (since I had disconnected dropbox during editing) there are no intermediate files to revert to. So it looks like Dropbox may sometimes corrupt a file geodatabase.
– Stephen Lead
May 14 '14 at 10:28
add a comment |
2
You quote: 'you might not notice any problems until you access a certain part of the new copy a few days later' scares the crap out of me. It would probably be way smarter to use shapefiles instead of a GeoDatabase.
– Cody Brown
Aug 21 '13 at 15:33
I have been using SkyDrive to sync file gdbs without issue for almost a year now. When we have proxy issues which sometimes means I can't access Microsoft services, then I normally close my SkyDrive and only allow it to sync again once access has been restored. I have edited toolboxes with models and scripts when my SkyDrive is open but not syncing properly, and I have not lost anything yet.
– Cindy Jayakumar
Aug 23 '13 at 6:08
1
FWIW I was editing a complex polygon layer in a file GDB so I disconnected Dropbox while I was working on it. I reconnected Dropbox after editing, so the files were backed up. 24 hours later, all my edits are gone, and (since I had disconnected dropbox during editing) there are no intermediate files to revert to. So it looks like Dropbox may sometimes corrupt a file geodatabase.
– Stephen Lead
May 14 '14 at 10:28
2
2
You quote: 'you might not notice any problems until you access a certain part of the new copy a few days later' scares the crap out of me. It would probably be way smarter to use shapefiles instead of a GeoDatabase.
– Cody Brown
Aug 21 '13 at 15:33
You quote: 'you might not notice any problems until you access a certain part of the new copy a few days later' scares the crap out of me. It would probably be way smarter to use shapefiles instead of a GeoDatabase.
– Cody Brown
Aug 21 '13 at 15:33
I have been using SkyDrive to sync file gdbs without issue for almost a year now. When we have proxy issues which sometimes means I can't access Microsoft services, then I normally close my SkyDrive and only allow it to sync again once access has been restored. I have edited toolboxes with models and scripts when my SkyDrive is open but not syncing properly, and I have not lost anything yet.
– Cindy Jayakumar
Aug 23 '13 at 6:08
I have been using SkyDrive to sync file gdbs without issue for almost a year now. When we have proxy issues which sometimes means I can't access Microsoft services, then I normally close my SkyDrive and only allow it to sync again once access has been restored. I have edited toolboxes with models and scripts when my SkyDrive is open but not syncing properly, and I have not lost anything yet.
– Cindy Jayakumar
Aug 23 '13 at 6:08
1
1
FWIW I was editing a complex polygon layer in a file GDB so I disconnected Dropbox while I was working on it. I reconnected Dropbox after editing, so the files were backed up. 24 hours later, all my edits are gone, and (since I had disconnected dropbox during editing) there are no intermediate files to revert to. So it looks like Dropbox may sometimes corrupt a file geodatabase.
– Stephen Lead
May 14 '14 at 10:28
FWIW I was editing a complex polygon layer in a file GDB so I disconnected Dropbox while I was working on it. I reconnected Dropbox after editing, so the files were backed up. 24 hours later, all my edits are gone, and (since I had disconnected dropbox during editing) there are no intermediate files to revert to. So it looks like Dropbox may sometimes corrupt a file geodatabase.
– Stephen Lead
May 14 '14 at 10:28
add a comment |
If you need to have multiple PCs accessing files, it would seem to make more sense to migrate to a true spatial server setup rather than an ad hoc one using Dropbox. This could be either a hosted server running ArcGIS or a PostGIS database running on a cloud web service like AWS. My preference would be for the latter. It'll take a little more work to set up, but you can define your own relations and scale up to as many computers as you need (depending on how you configure AWS), all for much less than you'd be paying Esri.
There are several SO posts that discuss PostGIS and ArcGIS here and here. The second one seems to have more up-to-date information in the comments below the accepted answer.
Without something more robust handling the traffic from the different clients, you're probably going to be constantly dealing with data corruption issues. At the very least, if you moved to shapefiles, you should check out GeoGit. This would give you basic revision control so shapefiles edited on one computer can be merged with edits from another without being completely overwritten.
edited yesterday
PolyGeo♦
53.8k1781245
answered Aug 22 '13 at 19:46
Richard DRichard D
42028
Web servers are a great way to manage GIS Data across several PC's. In fact I agree, it's probably the best way. The issue with it is bandwidth and processing speed. Usually the internet speed offsite is high enough to work with the data. But, if I want to do some heavy analysis of the data, there is no way I want to be pulling it across the web. That's why I prefer a solution like Dropbox. I show up onsite and the GeoDatabases are all up to date and stored locally on the PC. Great way to work.
– Cody Brown
Aug 22 '13 at 20:10
3
@codybrown But you're still dealing with pushing data over the Internet with Dropbox, not to mention the sync issues you're getting, probably because it's taking so long to update from one client to another. At least a server would be optimized for that task. If you're really just looking to have a storage solution for data while you work from home, work, or client site, then a large removal hard drive is the only way to avoid latency and potential sync conflicts, assuming you're the only one using the data at any given time.
– Richard D
Aug 22 '13 at 21:41
add a comment |
If you need to have multiple PCs accessing files, it would seem to make more sense to migrate to a true spatial server setup rather than an ad hoc one using Dropbox. This could be either a hosted server running ArcGIS or a PostGIS database running on a cloud web service like AWS. My preference would be for the latter. It'll take a little more work to set up, but you can define your own relations and scale up to as many computers as you need (depending on how you configure AWS), all for much less than you'd be paying Esri.
There are several SO posts that discuss PostGIS and ArcGIS here and here. The second one seems to have more up-to-date information in the comments below the accepted answer.
Without something more robust handling the traffic from the different clients, you're probably going to be constantly dealing with data corruption issues. At the very least, if you moved to shapefiles, you should check out GeoGit. This would give you basic revision control so shapefiles edited on one computer can be merged with edits from another without being completely overwritten.
edited yesterday
PolyGeo♦
53.8k1781245
answered Aug 22 '13 at 19:46
Richard DRichard D
42028
Web servers are a great way to manage GIS Data across several PC's. In fact I agree, it's probably the best way. The issue with it is bandwidth and processing speed. Usually the internet speed offsite is high enough to work with the data. But, if I want to do some heavy analysis of the data, there is no way I want to be pulling it across the web. That's why I prefer a solution like Dropbox. I show up onsite and the GeoDatabases are all up to date and stored locally on the PC. Great way to work.
– Cody Brown
Aug 22 '13 at 20:10
3
@codybrown But you're still dealing with pushing data over the Internet with Dropbox, not to mention the sync issues you're getting, probably because it's taking so long to update from one client to another. At least a server would be optimized for that task. If you're really just looking to have a storage solution for data while you work from home, work, or client site, then a large removal hard drive is the only way to avoid latency and potential sync conflicts, assuming you're the only one using the data at any given time.
– Richard D
Aug 22 '13 at 21:41
add a comment |
If you need to have multiple PCs accessing files, it would seem to make more sense to migrate to a true spatial server setup rather than an ad hoc one using Dropbox. This could be either a hosted server running ArcGIS or a PostGIS database running on a cloud web service like AWS. My preference would be for the latter. It'll take a little more work to set up, but you can define your own relations and scale up to as many computers as you need (depending on how you configure AWS), all for much less than you'd be paying Esri.
There are several SO posts that discuss PostGIS and ArcGIS here and here. The second one seems to have more up-to-date information in the comments below the accepted answer.
Without something more robust handling the traffic from the different clients, you're probably going to be constantly dealing with data corruption issues. At the very least, if you moved to shapefiles, you should check out GeoGit. This would give you basic revision control so shapefiles edited on one computer can be merged with edits from another without being completely overwritten.
edited yesterday
PolyGeo♦
53.8k1781245
answered Aug 22 '13 at 19:46
Richard DRichard D
42028
If you need to have multiple PCs accessing files, it would seem to make more sense to migrate to a true spatial server setup rather than an ad hoc one using Dropbox. This could be either a hosted server running ArcGIS or a PostGIS database running on a cloud web service like AWS. My preference would be for the latter. It'll take a little more work to set up, but you can define your own relations and scale up to as many computers as you need (depending on how you configure AWS), all for much less than you'd be paying Esri.
There are several SO posts that discuss PostGIS and ArcGIS here and here. The second one seems to have more up-to-date information in the comments below the accepted answer.
Without something more robust handling the traffic from the different clients, you're probably going to be constantly dealing with data corruption issues. At the very least, if you moved to shapefiles, you should check out GeoGit. This would give you basic revision control so shapefiles edited on one computer can be merged with edits from another without being completely overwritten.
edited yesterday
PolyGeo♦
53.8k1781245
answered Aug 22 '13 at 19:46
Richard DRichard D
42028
edited yesterday
PolyGeo♦
53.8k1781245
edited yesterday
PolyGeo♦
53.8k1781245
edited yesterday
PolyGeo♦
53.8k1781245
53.8k1781245
answered Aug 22 '13 at 19:46
Richard DRichard D
42028
answered Aug 22 '13 at 19:46
Richard DRichard D
42028
answered Aug 22 '13 at 19:46
Richard DRichard D
42028
42028
Web servers are a great way to manage GIS Data across several PC's. In fact I agree, it's probably the best way. The issue with it is bandwidth and processing speed. Usually the internet speed offsite is high enough to work with the data. But, if I want to do some heavy analysis of the data, there is no way I want to be pulling it across the web. That's why I prefer a solution like Dropbox. I show up onsite and the GeoDatabases are all up to date and stored locally on the PC. Great way to work.
– Cody Brown
Aug 22 '13 at 20:10
3
@codybrown But you're still dealing with pushing data over the Internet with Dropbox, not to mention the sync issues you're getting, probably because it's taking so long to update from one client to another. At least a server would be optimized for that task. If you're really just looking to have a storage solution for data while you work from home, work, or client site, then a large removal hard drive is the only way to avoid latency and potential sync conflicts, assuming you're the only one using the data at any given time.
– Richard D
Aug 22 '13 at 21:41
add a comment |
Web servers are a great way to manage GIS Data across several PC's. In fact I agree, it's probably the best way. The issue with it is bandwidth and processing speed. Usually the internet speed offsite is high enough to work with the data. But, if I want to do some heavy analysis of the data, there is no way I want to be pulling it across the web. That's why I prefer a solution like Dropbox. I show up onsite and the GeoDatabases are all up to date and stored locally on the PC. Great way to work.
– Cody Brown
Aug 22 '13 at 20:10
3
@codybrown But you're still dealing with pushing data over the Internet with Dropbox, not to mention the sync issues you're getting, probably because it's taking so long to update from one client to another. At least a server would be optimized for that task. If you're really just looking to have a storage solution for data while you work from home, work, or client site, then a large removal hard drive is the only way to avoid latency and potential sync conflicts, assuming you're the only one using the data at any given time.
– Richard D
Aug 22 '13 at 21:41
Web servers are a great way to manage GIS Data across several PC's. In fact I agree, it's probably the best way. The issue with it is bandwidth and processing speed. Usually the internet speed offsite is high enough to work with the data. But, if I want to do some heavy analysis of the data, there is no way I want to be pulling it across the web. That's why I prefer a solution like Dropbox. I show up onsite and the GeoDatabases are all up to date and stored locally on the PC. Great way to work.
– Cody Brown
Aug 22 '13 at 20:10
Web servers are a great way to manage GIS Data across several PC's. In fact I agree, it's probably the best way. The issue with it is bandwidth and processing speed. Usually the internet speed offsite is high enough to work with the data. But, if I want to do some heavy analysis of the data, there is no way I want to be pulling it across the web. That's why I prefer a solution like Dropbox. I show up onsite and the GeoDatabases are all up to date and stored locally on the PC. Great way to work.
– Cody Brown
Aug 22 '13 at 20:10
3
3
@codybrown But you're still dealing with pushing data over the Internet with Dropbox, not to mention the sync issues you're getting, probably because it's taking so long to update from one client to another. At least a server would be optimized for that task. If you're really just looking to have a storage solution for data while you work from home, work, or client site, then a large removal hard drive is the only way to avoid latency and potential sync conflicts, assuming you're the only one using the data at any given time.
– Richard D
Aug 22 '13 at 21:41
@codybrown But you're still dealing with pushing data over the Internet with Dropbox, not to mention the sync issues you're getting, probably because it's taking so long to update from one client to another. At least a server would be optimized for that task. If you're really just looking to have a storage solution for data while you work from home, work, or client site, then a large removal hard drive is the only way to avoid latency and potential sync conflicts, assuming you're the only one using the data at any given time.
– Richard D
Aug 22 '13 at 21:41
add a comment |
Thanks for contributing an answer to Geographic Information Systems Stack Exchange!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function ()
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fgis.stackexchange.com%2fquestions%2f69307%2fusing-dropbox-to-sync-file-geodatabases%23new-answer', 'question_page');
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
3
I don't have a definitive answer, but I'd suggest that you're wise to be cautious. I don't believe that DropBox is intended for accessing a complex data format like a file geodatabase, which consists of multiple discrete files, on multiple PCs simultaneously. This post shows some potential issues to be aware of
– Stephen Lead
Aug 21 '13 at 15:18
@StephenLead I've been backing up the Dropbox data daily. I used Amazon Cloud Drive for about a year with no issues, but that is so slow for syncing. Dropbox works way quicker and simpler. I really hope that it can handle it.
– Cody Brown
Aug 21 '13 at 15:28
2
It is better to compress the file geodatabase before it is used to share resources.arcgis.com/en/help/main/10.2/index.html#//… it is not designed to be edited with more than one user at one time - lck files should prevent this.
– Mapperz♦
Aug 21 '13 at 16:31
Solid idea @Mapperz
– Cody Brown
Aug 21 '13 at 16:42