Table of Contents§

I need data exports to build tools around my personal data, and the actual process of exporting it from a silo is the first step.

After I export it I use it to build a ‘data mirror’.

Here I mostly keep the notes about the data I haven’t finished exporting.
The ones I have already/mostly finished are mentioned here:

* motivation§

Similar to backups.

`[2020-03-17]` "GitHub blocked me and all my libraries" https://news.ycombinator.com/item?id=22593595 §

motivation for using gdpr/takeouts: convenient when you’re migrating off the service? don’t have to worry about regular exports§

`[2020-05-31]` Own your content on Social Media using the IndieWeb - YouTube [[dataliberation]]§

* implementation: goals/tips/practices§

`[2019-10-03]` another big goal is having little operational overhead. I’d rather set up a (potentially elaborate) system once and tthen never have to update it and think how it works [[exports]] [[infra]]§

`[2019-10-03]` that involves automatic ci [[ci]]§

`[2019-10-03]` continuous cloud sync [[cloud]]§

`[2019-10-03]` automation/cron jobs for orger [[dron]]§

`[2021-03-04]` Importance of agnostic exports: ofter you start backing up before you process the data§

synthetic style exports allow for defensive error handling – you can at least get data from the last state [[errors]]§

eh. maybe get rid of colored logs for export process? presumably no one would look at them often§

use submodule for common files, but release as a standalone package? I guess it’s the best of both§

`[2020-01-01]` ChromeDevTools/devtools-protocol: Chrome DevTools Protocol [[exports]] [[scrape]]§

https://github.com/ChromeDevTools/devtools-protocol

`[2020-04-19]` open files using utf-8 encoding (fixes #5) by miguelrochefort · Pull Request #6 · karlicoss/rexport§

apply this to export helper…

backup-wrapper is a more generic tool… basically running arb command and saving output with pattern§

thinking about data providers [[dataliberation]]§

Easiest option is just to have separate scripts to run regularly?

most users won’t care about keeping historic data? Or maybe not keeping data at all? jsut provide lambda?§

so the backup script could provide TODO

most users won’t have cron set up?§

so need a way to trigger backup from promnesia indexer itself? Fairly easy to achieve as it’s all just python code?

to backup, use some python patternt library?§

example how it could work:§

in promnesia

def index_reddit():
    from exporters.reddit import export
    # TODO?
    return

dashboard for tokens + expose json or something so any language can have bindings [[infra]] [[exports]]§

might be annoying to implement token retrieval on JS only?

`[2020-04-12]` add this to myinfra repository??§

`[2020-05-27]` dunno, I’m a bit tired and not as motivated to build it… but could post so someone else picks up [[toblog]]§

ok, exposing a stream is sort of good? and then filtering? makes it easier to use synthetic exports [[hpi]] [[exports]] [[dal]]§

* twitter [[twitter]]§

Twitter is a big pain in the ass, they’ve become very hostile towards API access.
Even the archives are somewhat incomplete (e.g. favorites lack some metadata).

E.g. from Apply for API — Twitter Developers

Be thorough
We need to completely understand your use case before we can approve it. So, please include as much detail as possible in your application.

hmm. links that you get through search or API are shortened? [[linkrot]] [[twitter]] [[twint]]§

`[2020-04-28]` shit.. also RTs are shortened?? so I need to get retweets properly?§

`[2021-01-19]` bisguzar/twitter-scraper: Scrape the Twitter Frontend API without authentication. [[twitter]] [[exports]]§

twint itself should work as incremental export… and then DAL should combine [[twint]]§

Even though Twint uses db, they seem to treat is as a temporary storage, so the schema might change.
I’m also not super convinced by how reliable the code is (from quick glance), so would worry about data loss.

`[2019-07-28]` jonbakerfish/TweetScraper: TweetScraper is a simple crawler/spider for Twitter Search without using API [[twitter]]§

`[2021-02-09]` doesn’t work, this error :( https://github.com/bisguzar/twitter-scraper/issues/168 §

talon databases (lots of them!) [[hpi]] [[android]]§

`[2019-07-29]` taspinar/twitterscraper: Scrape Twitter for Tweets §

One of the bigger disadvantages of the Search API is that you can only access Tweets written in the past 7 days. This is a major bottleneck for anyone looking for older past data to make a model from. With TwitterScraper there is no such limitation.

`[2021-02-09]` https://github.com/taspinar/twitterscraper/issues/344 broken as well§

err, new twitter exports are half gig each?§

twint: possibly missing reply things (with ‘at’) [[twint]] [[hpi]]§

compare tw-before.org (twint) and tw-after.org (twidump) in views
retweets in twint are def missing

* hackernews [[hackernews]]§

`[2020-04-05]` Our plan is for the next version of HN’s API to simply serve a JSON version of e… | Hacker News [[hackernews]]§

https://news.ycombinator.com/item?id=22788526

Our plan is for the next version of HN's API to simply serve a JSON version of every page. I'm hoping to get to that this year.

`[2020-04-07]` Profile: karlicoss | Hacker News§

https://news.ycombinator.com/user?id=karlicoss

user:	karlicoss
created:	August 25, 2016
karma:	757

capture HN karma? maybe on all comments

`[2020-04-29]` need to mirror HN… [[hackernews]] [[exports]]§

materialistic – ‘read’ table [[promnesia]]§

could also have ‘exact’ time notion and ‘approximate’ time – when it’s guessed from the file timestamp etc

`[2021-03-05]` it’s impresive that pretty much every tool for exporting has some flaws [[hackernews]]§

don’t have ci

https://github.com/davenicoll/hackernews
- doesn’t even have main??
https://github.com/romaintailhurat/hns
- uses pickle??
https://github.com/amjd/HN-Saved-Links-Export
- too defensive
- writes to stdout
- can’t be used as API

HN data provider [[hpi]] [[orger]] [[promnesia]]§

https://github.com/HackerNews/API
https://hacker-news.firebaseio.com/v0/user/karlicoss.json?print=pretty – get user data
extract ‘submitted’
https://hacker-news.firebaseio.com/v0/item/25971799.json?print=pretty – comment
https://hacker-news.firebaseio.com/v0/item/25971380.json?print=pretty – type: "story"
dunno if useful to keep scores over time?
not sure if should dump everything in a single json? or split by files?
can change later I guess

* google takeout/other google data [[takeout]]§

Google Takeout doesn’t have a proper API, and periodic expots are kind of annoying… would be good to automate them.

Another difficulty is that the data seems to have a certain retention,
so you can’t just take the latest takeout, for some data you need to merge all of them.

wonder if it’s possible to get watch position? [[takeout]] [[youtube]]§

`[2019-09-28]` life-vault/selenium_takeout.py at master · ThorbenJensen/life-vault https://github.com/ThorbenJensen/life-vault/blob/master/src/takeout/selenium_takeout.py §

automating google drive [[takeout]] [[backup]] [[exports]]§

ocamlfuse + script to move to desired location
basically that only requires you to request new archive occasionally

automate google takeouts?§

maybe release my module for 2FA separately?
https://github.com/ThorbenJensen/life-vault/blob/master/src/takeout/selenium_takeout.py

`[2021-01-10]` Hypothesis [[takeout]]§

Seriously, check out ratarmount if you haven't. Since the Google Takeout spans multiple 50GB tgz files (I'm at ~14, not including Google Drive in the takeout), ratarmount is brilliant. It merges all of the tgz contents into a single folder structure so /path/a/1.jpg and /path/a/1.json might be in different tgz folders but are mounted in to the same folder.

could sync mini-takeouts? with only necessary stuff picked from them [[takeout]]§

youtube watch history – should be accumulated from multiple takeouts [[youtube]]§

`[2019-06-11]` eh, recompressing to .tar.xz only saved 100 mb [[takeout]]§

ugh, also when it’s too large, they split archive in two [[takeout]]§

also disappearing Disover/Myacvitiy?? [[takeout]]§

20180807 My Activity/Discover/MyActivity.html                                    20190523 20181015 My Activity/Discover/MyActivity.html                                    20190522 20181213 My Activity/Discover/MyActivity.html                                    20200122

`[2020-04-23]` I’ve found Google Takeouts to silently remove old data | beepb00p §

huh, so with my script to search takeout duplicates, I’ve figured out that from 2015 at least Search/MyActivity.html hasn’t been erased? interesting
but looks like Chrome/MyActivity.html still being removed

`[2020-04-24]` Takeout/My Activity/Search data is limited to last 10 years. Please remove limit - Google Search Community §

Takeout/My Activity/Search data is limited to last 10 years. Please remove limit

`[2020-04-29]` > I’ve already pulled down my 2-300GB Google Photos archive How? I’ve tried sev… | Hacker News §

cuu508 1 hour ago [-]
Takeout doesn't work in practice for bigger collections (archive creation routinely fails, timeouts while downloading, 50GB max size results in many splits)
I've used this 3rd party tool and it worked OK: https://github.com/gilesknap/gphotos-sync/
geekgonecrazy 1 hour ago [-]
I forgot to mention this. But yes the export failed several dozen times. I believe I ended up doing in chunks. It was hard to get them off

`[2020-05-04]` I replied to a similar point about hashing here - https://news.ycombinator.com/i… | Hacker News§

You're correct that the methods I described are a far cry from actually guaranteeing that the backup has no errors. In the same way that a unit test doesn't prove code is error-free, but _can_ justify increased confidence in the code, I'm interested in techniques that can justify increased confidence in my backups. Particularly in cases where I don't have direct access to the original data, and where exhaustively checking the data manually is too time-consuming to be worth it.

yes!

`[2020-01-01]` perkeep/gphotos-cdp: This program uses the Chrome DevTools Protocol to drive a Chrome session that downloads your photos stored in Google Photos. [[scrape]]§

https://github.com/perkeep/gphotos-cdp

In our original Perkeep issue, @bradfitz said that we might have to give up on APIs and resort to scraping, noting that the Chrome DevTools Protocol makes this pretty easy.

`[2019-06-28]` After hoarding over 50k YouTube videos, here is the youtube-dl command I settled on. : DataHoarder§

https://www.reddit.com/r/DataHoarder/comments/c6fh4x/after_hoarding_over_50k_youtube_videos_here_is/

After hoarding over 50k YouTube videos, here is the youtube-dl command I settled on.

`[2020-01-01]` perkeep/gphotos-cdp: This program uses the Chrome DevTools Protocol to drive a Chrome session that downloads your photos stored in Google Photos. https://github.com/perkeep/gphotos-cdp §

we'd like our photos mirrored in seconds or minutes, not weeks.

* emfit sleep tracker [[emfit]]§

Emfit QS is my sleep tracker.

`[2018-08-18]` Emfit has local API; would be nice to use it… [[emfit]]§

https://gist.github.com/harperreed/9d063322eb84e88bc2d0580885011bdd
https://gist.github.com/karlicoss/3361f6a239048a451daa2a02982ee180

`[2020-09-11]` sanielfishawy/emfit_data_getter: Gets heart rate and respiration rate from an Emfit QS device on the same local network. [[emfit]]§

`[2019-12-17]` downloadEmfitAPI.py https://gist.github.com/vanne02135/6901cc2b92315881080d0ce0f07c1a17 §

ugh. maybe autorefresh the token? Fuckig hell. [[emfit]]§

`[2021-02-06]` I think I ended up just using login + password. meh§

hmm, with emfit can code some sort of feedback tool which signals me to move when emfit loses signal [[emfit]]§

`[2020-05-29]` emfit API didn’t work for about three days straight… [[emfit]] [[backup]]§

`[2019-12-21]` samuelmr/emfit-qs: Unofficial Node client for Emfit QS§

https://github.com/samuelmr/emfit-qs

Exchange username and password to a token (expires in 7 days). You can also log in to qs.emfit.com and check the ´remember_token´ parameter passed to API calls (e.g. with developer tools of your browser).

* bluemaestro temperature sensor [[bluemaestro]]§

figure out bluemaestro, make sure all merged [[bluemaestro]]§

State "STRT" from "TODO" [2019-03-12]

actually wonder if I can connect it to computer? [[bluemaestro]]§

merge bluemaestros, plot separate environmental dashboard? [[dashboard]]§

automate, about how I back up bluemaestro data [[toblog]]§

`[2019-09-29]` yeah, could elaborate on backing up android data, could be quite generic? [[android]]§

* reddit [[reddit]]§

I think cool fact should just be converted into org mode from backups (merged!) but generally there is no point capturing them? [[reddit]]§

`[2019-09-10]` er, I guess for orger need to extract a simple reddit provider that just merges various timestamped backups?§

I guess just rely on bleanser instead after all? Just make it less spammy [[bleanser]] [[reddit]]§

Check for deleted favorites [[reddit]]§

State "STRT" from "TODO" [2019-03-23]

`[2019-08-25]` yep, it def happens; promnesia triggers it§

shit. need to bleanse reddit properly, otherwise looks like it’s too much data… [[reddit]]§

basically, just go through stuff that doesn’t exist anymore but was in favorites ever (and suppress errors for some of them) [[reddit]]§

`[2020-01-11]` Getting Started — PRAW 3.6.0 documentation [[reddit]]§

https://praw.readthedocs.io/en/v3.6.0/pages/getting_started.html#connecting-to-reddit >

Next question is, does a raw API call give the same results as the website? The API and website search engine run off of different indexes.

———-–— other data sources ———————-§

Podcast addict data§

`[2020-07-31]` alexattia/Maps-Location-History: Get, Concatenate and Process you location history from Google Maps TimeLine [[location]] [[timeline]] [[qs]]§

In order to export processed data from Google Maps website from a python script, you need to get your actual cookie.

fuck me! it actually exports kml files

ok, so need to preserve all (incl.older) versions of notebooks? dunno feels a bit excessive [[timeline]] [[remarkable]]§

`[2020-10-25]` Garmin Connect [[garmin]]§

Looks like you experienced a time change. This may cause some inaccuracies with today's data.

jesus!

`[2020-12-30]` Notice: This project is unmaintained · Issue #613 · fbchat-dev/fbchat [[facebook]]§

Need my email mirrored [[email]]§

`[2019-03-12]` I guess I want continuous. Not sure how to achieve that, perhaps some mail client in the background?§

`[2019-06-13]` joeyates/imap-backup: Backup GMail (or other IMAP) accounts to disk [[email]]§

https://github.com/joeyates/imap-backup

Bandcamp history§

`[2020-12-13]` https://bandcamp.com/developer no listening history though…§

hmm memrise personal data request is neat! It’s got all you training sessions + learned words and phrases [[publish]]§

do a full remarkable backup too? [[remarkable]]§

`[2020-11-27]` [tech:ssh reMarkableWiki]§

# the xochitl binary, if you plan on replacing or modifying it in any way
scp root@remarkable:/usr/bin/xochitl remarkable-backup/

better docs on what to do on expiry [[monzo]]§

Traceback (most recent call last):
  File "pymonzo/monzo_api.py", line 209, in _get_response
    raise TokenExpiredError
oauthlib.oauth2.rfc6749.errors.TokenExpiredError: (token_expired

this is how token looks like after:

modified: .pymonzo-token
{
    "code": "internal_service",
    "message": "An error occurred processing the request"
}

huh, thriva uses an api…§

call history from my old(er?) phones? (e.g. nokia)§

increase sample rate to 10 seconds maybe? [[arbtt]]§

process old ‘backups’ repo?§

reading hr data [[wahoo]]§

import fitparse
ff = fitparse.FitFile('2020-10-02-161142-TICKR X 076C-1601655102-0.fit')
[m.get_value('timestamp') for m in ff.messages]

NOTE: not all messages are hr messages, there is also some metadata etc.
https://github.com/perrygeo/graph-kickr/blob/master/app.py

`[2020-10-09]` also tried gpsbabel, but it resulted in no data… weird§

gpsbabel -i garmin_fit,allpoints=1 -f '2020-10-02-161142-TICKR X 076C-1601655102-0.fit' -o unicsv -F res.csv

`[2019-04-08]` python - Steam API get historical player count of specific game - Stack Overflow§

https://stackoverflow.com/questions/45983820/steam-api-get-historical-player-count-of-specific-game
There is no Steam Web API method for historical player count of a specific game.

Feedbin starred stuff§

`[2019-07-14]` fabianonline/telegram_backup: Java app to download all your telegram data.§

https://github.com/fabianonline/telegram_backup

Use --with-supergroups and / or --with-channels to also download all messages from the supergroups / channels you have joined that have been active in the last time.

eh, should include older account? compare oldest and one of newer files.. [[monzo]]§

myshows: hmm, so looks like api v 1.8 is deprecated, for api 2.0 I’d need to email them. can just use raw jsons from existing backup script§

compress chrome histories? would require backup script to compress it I suppose… maybe just go through them regularly and recompress§

bookmarks limit through api??? [[instapaper]]§

`[2020-01-04]` need to check historic exports and figure it out§

gpslogger – add to backup checker?? [[location]]§

`[2020-10-03]` Statify: Pull your playlist and listening data from the Spotify API to a Sqlite database /r/coolgithubprojects§

monzo export: make sure it works with original repo.. [[exports]] [[monzo]]§

`[2019-12-29]` huh, actually the problem might be in saving _token variable?§

`[2019-12-29]` instead could just print it from disk? maybe even that is unnecessary?§

ugh, need to retrieve pinboard notes [[pinboard]] [[exports]]§

e.g. motivational example of API discovery; I just assumed they all would be retrieved https://api.pinboard.in/v1/notes/ID

`[2019-04-23]` feedbin/feedbin-api: Feedbin API Documentation [[feedbin]]§

https://github.com/feedbin/feedbin-api#readme

The base URL for all requests is https://api.feedbin.com/v2/ Only https is supported.
The Feedbin API uses HTTP Basic authentication
curl -u 'example@example.com:password' https://api.feedbin.com/v2/subscriptions.json

`[2020-11-27]` Success Stories · tcgoetz/GarminDB Wiki [[garmin]]§

`[2020-12-19]` Importing your Goodreads & Accessing them with Open Library’s APIs §

Importing your Goodreads & Accessing them with Open Library’s APIs

`[2020-06-24]` Telegram Now Lets You Export Your Chats, View Notification Exceptions | Technology News [[telegram]]§

get off the messages stored in old format and make sure nothing is missing, dedup? [[vk]]§

shit, they stopped you from accessing messages api. fuck. [[vk]]§

https://vk.com/wall-1_390510

`[2019-03-08]` that’s very generic trend. I think ultimately we just need better tools to scrape that§

Headspace stats [[timeline]]§

UserTimelineEntry?

.polar directory [[timeline]]§

`[2019-09-02]` vincaslt/memparse: A Memrise courses parser https://github.com/vincaslt/memparse §

skype call history?§

`[2020-04-19]` shit https://answers.microsoft.com/en-us/skype/forum/all/skype-api/e025d0f6-7ae4-4bc4-9d5a-b2d70136deab §

I regret to inform you but we do not have API or a program in Skype that lets you export your chat history that will include dates, timestamps etc.

amazon orders history§

State "STRT" from "TODO" [2019-02-23]

`[2018-05-04]` ugh, order history report is broken for the UK version :( https://www.amazon.co.uk/gp/help/customer/display.html?nodeId=202119330 wrote to support§

https://www.amazon.co.uk/gp/b2b/reports
Then could connect to drebedengi and add comments (even with breakdown)

`[2019-02-23]` backups/amazon§

State "STRT" from [2019-02-23]

ugh, bookmarks method in api is not exhaustive (elif item.get("type") == ‘bookmark’) [[instapaper]]§

`[2019-04-01]` Polar AccessLink Api Daily Activity Goal /r/Polarfitness§

The API. You do need a session cookie for it and I didn't find an official documentation. The cookie can be retrieved by mimicking their login form. If you do have specific questions you can send me a DM but basically you just need to copy the requests their web app is making.

just reuse files dir? def no harm in it [[telegram]]§

blinkist: scrape off my highlights§

https://www.blinkist.com/en/nc/highlights

`[2019-08-13]` eh, just copy responses manually?§

`[2019-08-13]` huh, actually if webdriver could eavesdrop on json responses would be perfect§

`[2019-08-13]` post in on github… [[toblog]]§

export bitbucket§

feedbin§

`[2020-03-05]` signalnerve/roam-backup: Automated Roam Research backups using GitHub Actions and AWS S3§

https://github.com/signalnerve/roam-backup

To use it, just fork this repo and add the following secrets to your repo (naming must match!):
    roamEmail
    roamPasswor

——————————————————————————§

`[2020-02-03]` Data lake - Wikipedia [[dal]] [[exports]]§

https://en.wikipedia.org/wiki/Data_lake

Table of Contents§

related§

. [[silo]]§

. [[infra]]§

. [[backup]]§

. [[dataliberation]]§

* motivation§

[2020-03-17] "GitHub blocked me and all my libraries" https://news.ycombinator.com/item?id=22593595§

motivation for using gdpr/takeouts: convenient when you’re migrating off the service? don’t have to worry about regular exports§

[2020-05-31] Own your content on Social Media using the IndieWeb - YouTube [[dataliberation]]§

* implementation: goals/tips/practices§

[2019-10-03] another big goal is having little operational overhead. I’d rather set up a (potentially elaborate) system once and tthen never have to update it and think how it works [[exports]] [[infra]]§

[2019-10-03] that involves automatic ci [[ci]]§

[2019-10-03] continuous cloud sync [[cloud]]§

[2019-10-03] automation/cron jobs for orger [[dron]]§

[2021-03-04] Importance of agnostic exports: ofter you start backing up before you process the data§

recommend checking the database to make sure it’s got specific things you need [[backupchecker]]§

synthetic style exports allow for defensive error handling – you can at least get data from the last state [[errors]]§

eh. maybe get rid of colored logs for export process? presumably no one would look at them often§

use submodule for common files, but release as a standalone package? I guess it’s the best of both§

[2020-01-01] ChromeDevTools/devtools-protocol: Chrome DevTools Protocol [[exports]] [[scrape]]§

[2020-04-19] open files using utf-8 encoding (fixes #5) by miguelrochefort · Pull Request #6 · karlicoss/rexport§

backup-wrapper is a more generic tool… basically running arb command and saving output with pattern§

thinking about data providers [[dataliberation]]§

most users won’t care about keeping historic data? Or maybe not keeping data at all? jsut provide lambda?§

most users won’t have cron set up?§

to backup, use some python patternt library?§

example how it could work:§

dashboard for tokens + expose json or something so any language can have bindings [[infra]] [[exports]]§

[2020-04-12] add this to myinfra repository??§

[2020-05-27] dunno, I’m a bit tired and not as motivated to build it… but could post so someone else picks up [[toblog]]§

ok, exposing a stream is sort of good? and then filtering? makes it easier to use synthetic exports [[hpi]] [[exports]] [[dal]]§

* twitter [[twitter]]§

hmm. links that you get through search or API are shortened? [[linkrot]] [[twitter]] [[twint]]§

[2020-04-28] shit.. also RTs are shortened?? so I need to get retweets properly?§

[2021-01-19] bisguzar/twitter-scraper: Scrape the Twitter Frontend API without authentication. [[twitter]] [[exports]]§

twint itself should work as incremental export… and then DAL should combine [[twint]]§

[2019-07-28] jonbakerfish/TweetScraper: TweetScraper is a simple crawler/spider for Twitter Search without using API [[twitter]]§

[2021-02-09] doesn’t work, this error :( https://github.com/bisguzar/twitter-scraper/issues/168§

talon databases (lots of them!) [[hpi]] [[android]]§

[2019-07-29] taspinar/twitterscraper: Scrape Twitter for Tweets§

[2021-02-09] https://github.com/taspinar/twitterscraper/issues/344 broken as well§

err, new twitter exports are half gig each?§

twint: possibly missing reply things (with ‘at’) [[twint]] [[hpi]]§

* hackernews [[hackernews]]§

[2020-04-05] Our plan is for the next version of HN’s API to simply serve a JSON version of e… | Hacker News [[hackernews]]§

[2020-04-07] Profile: karlicoss | Hacker News§

[2020-04-29] need to mirror HN… [[hackernews]] [[exports]]§

materialistic – ‘read’ table [[promnesia]]§

[2021-03-05] it’s impresive that pretty much every tool for exporting has some flaws [[hackernews]]§

HN data provider [[hpi]] [[orger]] [[promnesia]]§

* google takeout/other google data [[takeout]]§

wonder if it’s possible to get watch position? [[takeout]] [[youtube]]§

automating login & downloading§

[2019-09-28] life-vault/seleniumtakeout.py at master · ThorbenJensen/life-vault https://github.com/ThorbenJensen/life-vault/blob/master/src/takeout/selenium_takeout.py§

automating google drive [[takeout]] [[backup]] [[exports]]§

automate google takeouts?§

[2021-01-10] Hypothesis [[takeout]]§

could sync mini-takeouts? with only necessary stuff picked from them [[takeout]]§

youtube watch history – should be accumulated from multiple takeouts [[youtube]]§

[2019-06-11] eh, recompressing to .tar.xz only saved 100 mb [[takeout]]§

ugh, also when it’s too large, they split archive in two [[takeout]]§

also disappearing Disover/Myacvitiy?? [[takeout]]§

[2020-04-23] I’ve found Google Takeouts to silently remove old data | beepb00p§

[2020-04-24] Takeout/My Activity/Search data is limited to last 10 years. Please remove limit - Google Search Community§

[2020-04-29] > I’ve already pulled down my 2-300GB Google Photos archive How? I’ve tried sev… | Hacker News§

[2020-05-04] I replied to a similar point about hashing here - https://news.ycombinator.com/i… | Hacker News§

[2020-01-01] perkeep/gphotos-cdp: This program uses the Chrome DevTools Protocol to drive a Chrome session that downloads your photos stored in Google Photos. [[scrape]]§

[2019-06-28] After hoarding over 50k YouTube videos, here is the youtube-dl command I settled on. : DataHoarder§

[2020-01-01] perkeep/gphotos-cdp: This program uses the Chrome DevTools Protocol to drive a Chrome session that downloads your photos stored in Google Photos. https://github.com/perkeep/gphotos-cdp§