Skip to content

Migrate LevelDB data out of ~/Documents #4609

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 51 commits into from
Jan 17, 2020
Merged

Conversation

wilhuff
Copy link
Contributor

@wilhuff wilhuff commented Jan 7, 2020

This moves iOS and macOS user data from NSDocumentDirectory to NSApplicationSupportDirectory, adding a pre-open migration step that:

  • Moves old data into the new location
  • Cleans up any resulting empty directories in the old Documents folder
  • Stops before removing the Documents folder itself (see tests)

Note that other platforms don't need a migration:

  • tvOS was already storing documents in NSCachesDirectory
  • Linux, Windows, and other UNIX haven't been publicly released or aren't changing.

See individual commits for details. Most notably this adds some additional filesystem manipulation routines to our (sadly) growing library.

Fixes #843.

wilhuff added 30 commits January 2, 2020 17:34
The spec tests aren't actually leaking, but they accumulate enough
garbage without clearing the autorelease pool that Instruments
complains.
ARC does not release temporaries when an Objective-C exception is
thrown. Use C++ equivalents directly to avoid leaks as detected by
Instruments.
Any previous value must be freed before the next one is assigned.
The single by-value implementation wasn't preventing the compiler
generated standard versions from being generated which was causing
leaks.
It's possible for the Executor to immediately execute the given block,
at which point the block might observe the async queue in a
not-shutting-down state.
After shutdown, the AsyncQueue silently drops Enqueue requests which
prevents these blocks from being deleted.
This avoids races between Shutdown and Complete and prevents ownership
ambiguity.
Use util::LogIsLoggable(util::kLogLevelDebug) directly.
  * Make methods const where possible
  * Make key() and value() return const std::string&, allowing callers
    to create their own const references
iOS: from Documents/firestore to Library/Application Support/firestore.
macOS: .firestore to Library/Application Support/firestore.
wilhuff added a commit that referenced this pull request Jan 9, 2020
This change does not yet change all the classes that use the filesystem
to take Filesystem as a dependency. Those changes will come in follow-up
PRs.

This change is motivated by feedback in #4609. There I had created a
complicated interface to LevelDbOpener just so that different migration
scenarios could be tested, when in reality, I just needed to be able to
plumb through different values for the result of AppDataDir and related
functions.
wilhuff added a commit that referenced this pull request Jan 9, 2020
This change does not yet change all the classes that use the filesystem
to take Filesystem as a dependency. Those changes will come in follow-up
PRs.

This change is motivated by feedback in #4609. There I had created a
complicated interface to LevelDbOpener just so that different migration
scenarios could be tested, when in reality, I just needed to be able to
plumb through different values for the result of AppDataDir and related
functions.
wilhuff added a commit that referenced this pull request Jan 10, 2020
This change does not yet change all the classes that use the filesystem
to take Filesystem as a dependency. Those changes will come in follow-up
PRs.

This change is motivated by feedback in #4609. There I had created a
complicated interface to LevelDbOpener just so that different migration
scenarios could be tested, when in reality, I just needed to be able to
plumb through different values for the result of AppDataDir and related
functions.
@wilhuff wilhuff changed the base branch from wilhuff/01-cleanup to master January 10, 2020 17:27
@wilhuff wilhuff force-pushed the wilhuff/02-leveldb-migration branch from 2432d88 to 6783f11 Compare January 10, 2020 19:31
Copy link
Contributor

@mikelehen mikelehen left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM with minor suggestions / nits. Feel free to push back / ignore the nits.

@wilhuff wilhuff assigned mikelehen and unassigned wilhuff Jan 12, 2020
@mikelehen mikelehen assigned wilhuff and unassigned mikelehen Jan 12, 2020
@wilhuff wilhuff merged commit ecb26bf into master Jan 17, 2020
@wilhuff wilhuff deleted the wilhuff/02-leveldb-migration branch January 17, 2020 01:36
@firebase firebase locked and limited conversation to collaborators Feb 16, 2020
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

New Location for the LevelDB directory
3 participants