diff options
| author |   Unit 193 <unit193@unit193.net> | 2025-04-15 05:25:37 -0400 |
|---|---|---|
| committer | Unit 193 <unit193@unit193.net> | 2025-04-15 05:25:37 -0400 |
| commit | b830dc03b3b7c9dd119648e1be9c1145d56e096c (patch) | |
| tree | e9d03b6b4ab93990243c0038c20ada2464fa4072 | |
| parent | 662e5ac868a5c1a3e7bc95b37054b3a0ca4db74f (diff) | |
New upstream version 1.29.4.upstream/1.29.4
33 files changed, 340 insertions, 193 deletions
diff --git a/CHANGELOG.md b/CHANGELOG.md index 257f47b..d779ffa 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -1,43 +1,32 @@ -## 1.29.3 - 2025-03-29 +## 1.29.4 - 2025-04-13 ### Extractors #### Additions -- [danbooru] add `favgroup` extractor -- [imhentai] support `hentaienvy.com` and `hentaizap.com` ([#7192](https://github.com/mikf/gallery-dl/issues/7192) [#7218](https://github.com/mikf/gallery-dl/issues/7218)) +- [chevereto] support `imagepond.net` ([#7278](https://github.com/mikf/gallery-dl/issues/7278)) +- [webtoons] add `artist` extractor ([#7274](https://github.com/mikf/gallery-dl/issues/7274)) #### Fixes -- [bunkr] fix `filename` extraction ([#7237](https://github.com/mikf/gallery-dl/issues/7237)) -- [deviantart:stash] fix legacy `sta.sh` links ([#7181](https://github.com/mikf/gallery-dl/issues/7181)) -- [hitomi] fix extractors ([#7230](https://github.com/mikf/gallery-dl/issues/7230)) -- [mangapark] fix extractors ([#4999](https://github.com/mikf/gallery-dl/issues/4999) [#5883](https://github.com/mikf/gallery-dl/issues/5883) [#6507](https://github.com/mikf/gallery-dl/issues/6507) [#6908](https://github.com/mikf/gallery-dl/issues/6908) [#7232](https://github.com/mikf/gallery-dl/issues/7232)) -- [nozomi] fix extractors ([#7242](https://github.com/mikf/gallery-dl/issues/7242)) -- [patreon] include subdomains in `session_id` cookie check ([#7188](https://github.com/mikf/gallery-dl/issues/7188)) -- [patreon] do not match `/messages` URLs as creator ([#7187](https://github.com/mikf/gallery-dl/issues/7187)) -- [pinterest] handle `story_pin_static_sticker_block` blocks ([#7251](https://github.com/mikf/gallery-dl/issues/7251)) -- [sexcom] fix `gif` pin extraction ([#7239](https://github.com/mikf/gallery-dl/issues/7239)) -- [skeb] make exceptions when extracting posts non-fatal ([#7250](https://github.com/mikf/gallery-dl/issues/7250)) -- [zerochan] parse `JSON-LD` data ([#7178](https://github.com/mikf/gallery-dl/issues/7178)) +- [deviantart] fix `KeyError: 'has_subfolders'` ([#7272](https://github.com/mikf/gallery-dl/issues/7272) [#7337](https://github.com/mikf/gallery-dl/issues/7337)) +- [discord] fix `parent` keyword inconsistency ([#7341](https://github.com/mikf/gallery-dl/issues/7341) [#7353](https://github.com/mikf/gallery-dl/issues/7353)) +- [E621:pool] fix `AttributeError` ([#7265](https://github.com/mikf/gallery-dl/issues/7265) [#7344](https://github.com/mikf/gallery-dl/issues/7344)) +- [everia] fix/improve image extraction ([#7270](https://github.com/mikf/gallery-dl/issues/7270)) +- [gelbooru] fix video URLs ([#7345](https://github.com/mikf/gallery-dl/issues/7345)) +- [hentai2read] fix `AttributeError` exception for chapters without artist ([#7355](https://github.com/mikf/gallery-dl/issues/7355)) +- [issuu] fix extractors ([#7317](https://github.com/mikf/gallery-dl/issues/7317)) +- [kemonoparty] fix file paths with backslashes ([#7321](https://github.com/mikf/gallery-dl/issues/7321)) +- [readcomiconline] fix `issue` extractor ([#7269](https://github.com/mikf/gallery-dl/issues/7269) [#7330](https://github.com/mikf/gallery-dl/issues/7330)) +- [rule34xyz] update to API v2 ([#7289](https://github.com/mikf/gallery-dl/issues/7289)) +- [zerochan] fix `KeyError: 'author'` ([#7282](https://github.com/mikf/gallery-dl/issues/7282)) #### Improvements -- [arcalive] extend `gifs` option -- [deviantart] support multiple images for single posts ([#6653](https://github.com/mikf/gallery-dl/issues/6653) [#7261](https://github.com/mikf/gallery-dl/issues/7261)) -- [deviantart] add subfolder support ([#4988](https://github.com/mikf/gallery-dl/issues/4988) [#7185](https://github.com/mikf/gallery-dl/issues/7185) [#7220](https://github.com/mikf/gallery-dl/issues/7220)) -- [deviantart] match `/gallery/recommended-for-you` URLs ([#7168](https://github.com/mikf/gallery-dl/issues/7168) [#7243](https://github.com/mikf/gallery-dl/issues/7243)) -- [instagram] extract videos from `video_dash_manifest` data ([#6379](https://github.com/mikf/gallery-dl/issues/6379) [#7006](https://github.com/mikf/gallery-dl/issues/7006)) -- [mangapark] support mirror domains -- [mangapark] support v3 URLs ([#2072](https://github.com/mikf/gallery-dl/issues/2072)) -- [mastodon] support `/statuses` URLs ([#7255](https://github.com/mikf/gallery-dl/issues/7255)) -- [sexcom] support new-style `/gifs` and `/videos` URLs ([#7239](https://github.com/mikf/gallery-dl/issues/7239)) -- [subscribestar] detect redirects to `/age_confirmation_warning` pages -- [tiktok] add retry mechanism to rehydration data extraction ([#7191](https://github.com/mikf/gallery-dl/issues/7191)) -#### Metadata -- [bbc] extract more metadata ([#6582](https://github.com/mikf/gallery-dl/issues/6582)) -- [kemonoparty] extract `archives` metadata ([#7195](https://github.com/mikf/gallery-dl/issues/7195)) -- [kemonoparty] enable `username`/`user_profile` metadata by default -- [kemonoparty:discord] always provide `channel_name` metadata ([#7245](https://github.com/mikf/gallery-dl/issues/7245)) -- [sexcom] extract `date_url` metadata ([#7239](https://github.com/mikf/gallery-dl/issues/7239)) -- [subscribestar] extract `title` metadata ([#7219](https://github.com/mikf/gallery-dl/issues/7219)) +- [instagram] use Chrome `User-Agent` by default ([#6379](https://github.com/mikf/gallery-dl/issues/6379)) +- [pixiv] support `phixiv.net` URLs ([#7352](https://github.com/mikf/gallery-dl/issues/7352)) +- [tumblr] support URLs without subdomain ([#7358](https://github.com/mikf/gallery-dl/issues/7358)) +- [webtoons] download JPEG files in higher quality +- [webtoons] use a default 0.5-1.5s delay between requests ([#7329](https://github.com/mikf/gallery-dl/issues/7329)) +- [zzup] support `w.zzup.com` URLs ([#7327](https://github.com/mikf/gallery-dl/issues/7327)) ### Downloaders -- [ytdl] support processing inline HLS/DASH manifest data ([#6379](https://github.com/mikf/gallery-dl/issues/6379) [#7006](https://github.com/mikf/gallery-dl/issues/7006)) +- [ytdl] fix `KeyError: 'extractor'` exception when `ytdl` reports an error ([#7301](https://github.com/mikf/gallery-dl/issues/7301)) +### Post Processors +- [metadata] add `metadata-path` option ([#6582](https://github.com/mikf/gallery-dl/issues/6582)) +- [metadata] fix handling of empty directory paths ([#7296](https://github.com/mikf/gallery-dl/issues/7296)) +- [ugoira] preserve `extension` when using `"mode": "archive"` ([#7304](https://github.com/mikf/gallery-dl/issues/7304)) ### Miscellaneous -- [aes] simplify `block_count` calculation -- [common] add `subdomains` argument to `cookies_check()` ([#7188](https://github.com/mikf/gallery-dl/issues/7188)) -- [config] fix using the same key multiple times with `apply` ([#7127](https://github.com/mikf/gallery-dl/issues/7127)) -- [tests] implement expected failures +- [formatter] add `i` and `f` conversions ([#6582](https://github.com/mikf/gallery-dl/issues/6582)) @@ -1,6 +1,6 @@ Metadata-Version: 2.4 Name: gallery_dl -Version: 1.29.3 +Version: 1.29.4 Summary: Command-line program to download image galleries and collections from several image hosting sites Home-page: https://github.com/mikf/gallery-dl Download-URL: https://github.com/mikf/gallery-dl/releases/latest @@ -133,9 +133,9 @@ Standalone Executable Prebuilt executable files with a Python interpreter and required Python packages included are available for -- `Windows <https://github.com/mikf/gallery-dl/releases/download/v1.29.3/gallery-dl.exe>`__ +- `Windows <https://github.com/mikf/gallery-dl/releases/download/v1.29.4/gallery-dl.exe>`__ (Requires `Microsoft Visual C++ Redistributable Package (x86) <https://aka.ms/vs/17/release/vc_redist.x86.exe>`__) -- `Linux <https://github.com/mikf/gallery-dl/releases/download/v1.29.3/gallery-dl.bin>`__ +- `Linux <https://github.com/mikf/gallery-dl/releases/download/v1.29.4/gallery-dl.bin>`__ Nightly Builds @@ -77,9 +77,9 @@ Standalone Executable Prebuilt executable files with a Python interpreter and required Python packages included are available for -- `Windows <https://github.com/mikf/gallery-dl/releases/download/v1.29.3/gallery-dl.exe>`__ +- `Windows <https://github.com/mikf/gallery-dl/releases/download/v1.29.4/gallery-dl.exe>`__ (Requires `Microsoft Visual C++ Redistributable Package (x86) <https://aka.ms/vs/17/release/vc_redist.x86.exe>`__) -- `Linux <https://github.com/mikf/gallery-dl/releases/download/v1.29.3/gallery-dl.bin>`__ +- `Linux <https://github.com/mikf/gallery-dl/releases/download/v1.29.4/gallery-dl.bin>`__ Nightly Builds diff --git a/data/man/gallery-dl.1 b/data/man/gallery-dl.1 index 5b0e7e7..7eb34af 100644 --- a/data/man/gallery-dl.1 +++ b/data/man/gallery-dl.1 @@ -1,4 +1,4 @@ -.TH "GALLERY-DL" "1" "2025-03-29" "1.29.3" "gallery-dl Manual" +.TH "GALLERY-DL" "1" "2025-04-13" "1.29.4" "gallery-dl Manual" .\" disable hyphenation .nh diff --git a/data/man/gallery-dl.conf.5 b/data/man/gallery-dl.conf.5 index d032f25..dc11605 100644 --- a/data/man/gallery-dl.conf.5 +++ b/data/man/gallery-dl.conf.5 @@ -1,4 +1,4 @@ -.TH "GALLERY-DL.CONF" "5" "2025-03-29" "1.29.3" "gallery-dl Manual" +.TH "GALLERY-DL.CONF" "5" "2025-04-13" "1.29.4" "gallery-dl Manual" .\" disable hyphenation .nh .\" disable justification (adjust text to left margin only) @@ -472,6 +472,7 @@ response before \f[I]retrying\f[] the request. \f[I]soundgasm\f[], \f[I]urlgalleries\f[], \f[I]vk\f[], +\f[I]webtoons\f[], \f[I]weebcentral\f[], \f[I]xfolio\f[], \f[I]zerochan\f[] @@ -815,6 +816,8 @@ or a \f[I]list\f[] with IP and explicit port number as elements. .br * \f[I]"Patreon/72.2.28 (Android; Android 14; Scale/2.10)"\f[]: \f[I]patreon\f[] .br +* \f[I]"Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/134.0.0.0 Safari/537.36"\f[]: \f[I]instagram\f[] +.br * \f[I]"Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:LATEST) Gecko/20100101 Firefox/LATEST"\f[]: otherwise .IP "Description:" 4 @@ -6212,6 +6215,45 @@ Fetch extra submission metadata during gallery downloads. Note: This requires 1 additional HTTP request per submission. +.SS extractor.webtoons.quality +.IP "Type:" 6 +.br +* \f[I]integer\f[] +.br +* \f[I]string\f[] +.br +* \f[I]object\f[] (ext -> type) + + +.IP "Default:" 9 +\f[I]"original"\f[] + +.IP "Example:" 4 +.br +* 90 +.br +* "q50" +.br +* {"jpg": "q80", "jpeg": "q80", "png": false} + +.IP "Description:" 4 +Controls the quality of downloaded files by modifying URLs' \f[I]type\f[] parameter. + +\f[I]"original"\f[] +Download minimally compressed versions of JPG files +any \f[I]integer\f[] +Use \f[I]"q<VALUE>"\f[] as \f[I]type\f[] parameter for JPEG files +any \f[I]string\f[] +Use this value as \f[I]type\f[] parameter for JPEG files +any \f[I]object\f[] +Use the given values as \f[I]type\f[] parameter for URLs with the specified extensions +.br +- Set a value to \f[I]false\f[] to completely remove these extension's \f[I]type\f[] parameter +.br +- Omit an extension to leave its URLs unchanged +.br + + .SS extractor.weibo.gifs .IP "Type:" 6 .br @@ -7731,6 +7773,18 @@ files with, which will replace the original filename extensions. Note: \f[I]metadata.extension\f[] is ignored if this option is set. +.SS metadata.metadata-path +.IP "Type:" 6 +\f[I]string\f[] + +.IP "Example:" 4 +"_meta_path" + +.IP "Description:" 4 +Insert the path of generated files +into metadata dictionaries as the given name. + + .SS metadata.event .IP "Type:" 6 .br diff --git a/docs/gallery-dl.conf b/docs/gallery-dl.conf index 8ede568..b85a3e7 100644 --- a/docs/gallery-dl.conf +++ b/docs/gallery-dl.conf @@ -732,11 +732,13 @@ "api-key" : null, "metadata": false }, - "weebcentral": + "webtoons": { - "sleep-request": "0.5-1.5" + "sleep-request": "0.5-1.5", + + "quality": "original" }, - "xfolio": + "weebcentral": { "sleep-request": "0.5-1.5" }, @@ -751,6 +753,10 @@ "retweets" : false, "videos" : true }, + "xfolio": + { + "sleep-request": "0.5-1.5" + }, "ytdl": { "cmdline-args": null, diff --git a/gallery_dl.egg-info/PKG-INFO b/gallery_dl.egg-info/PKG-INFO index 4481e14..3d113ec 100644 --- a/gallery_dl.egg-info/PKG-INFO +++ b/gallery_dl.egg-info/PKG-INFO @@ -1,6 +1,6 @@ Metadata-Version: 2.4 Name: gallery_dl -Version: 1.29.3 +Version: 1.29.4 Summary: Command-line program to download image galleries and collections from several image hosting sites Home-page: https://github.com/mikf/gallery-dl Download-URL: https://github.com/mikf/gallery-dl/releases/latest @@ -133,9 +133,9 @@ Standalone Executable Prebuilt executable files with a Python interpreter and required Python packages included are available for -- `Windows <https://github.com/mikf/gallery-dl/releases/download/v1.29.3/gallery-dl.exe>`__ +- `Windows <https://github.com/mikf/gallery-dl/releases/download/v1.29.4/gallery-dl.exe>`__ (Requires `Microsoft Visual C++ Redistributable Package (x86) <https://aka.ms/vs/17/release/vc_redist.x86.exe>`__) -- `Linux <https://github.com/mikf/gallery-dl/releases/download/v1.29.3/gallery-dl.bin>`__ +- `Linux <https://github.com/mikf/gallery-dl/releases/download/v1.29.4/gallery-dl.bin>`__ Nightly Builds diff --git a/gallery_dl/downloader/ytdl.py b/gallery_dl/downloader/ytdl.py index 9d653b3..7a20dc2 100644 --- a/gallery_dl/downloader/ytdl.py +++ b/gallery_dl/downloader/ytdl.py @@ -200,6 +200,7 @@ class YoutubeDLDownloader(DownloaderBase): return None info_dict = { + "extractor": "", "id" : video_id, "title" : video_id, "formats" : fmts, diff --git a/gallery_dl/extractor/chevereto.py b/gallery_dl/extractor/chevereto.py index c9ccb7d..600d231 100644 --- a/gallery_dl/extractor/chevereto.py +++ b/gallery_dl/extractor/chevereto.py @@ -18,19 +18,23 @@ class CheveretoExtractor(BaseExtractor): directory_fmt = ("{category}", "{user}", "{album}",) archive_fmt = "{id}" - def __init__(self, match): - BaseExtractor.__init__(self, match) - self.path = match.group(match.lastindex) + def _init(self): + self.path = self.groups[-1] def _pagination(self, url): - while url: + while True: page = self.request(url).text for item in text.extract_iter( page, '<div class="list-item-image ', 'image-container'): - yield text.extr(item, '<a href="', '"') + yield text.urljoin(self.root, text.extr( + item, '<a href="', '"')) - url = text.extr(page, '<a data-pagination="next" href="', '" ><') + url = text.extr(page, 'data-pagination="next" href="', '"') + if not url: + return + if url[0] == "/": + url = self.root + url BASE_PATTERN = CheveretoExtractor.update({ @@ -42,6 +46,10 @@ BASE_PATTERN = CheveretoExtractor.update({ "root": "https://img.kiwi", "pattern": r"img\.kiwi", }, + "imagepond": { + "root": "https://imagepond.net", + "pattern": r"imagepond\.net", + }, }) diff --git a/gallery_dl/extractor/danbooru.py b/gallery_dl/extractor/danbooru.py index 741800c..06c31b9 100644 --- a/gallery_dl/extractor/danbooru.py +++ b/gallery_dl/extractor/danbooru.py @@ -282,10 +282,11 @@ class DanbooruPoolExtractor(DanbooruExtractor): example = "https://danbooru.donmai.us/pools/12345" def metadata(self): - return self._collection_metadata(self.groups[-1], "pool") + self.pool_id = self.groups[-1] + return self._collection_metadata(self.pool_id, "pool") def posts(self): - return self._collection_posts(self.groups[-1], "pool") + return self._collection_posts(self.pool_id, "pool") class DanbooruFavgroupExtractor(DanbooruExtractor): diff --git a/gallery_dl/extractor/deviantart.py b/gallery_dl/extractor/deviantart.py index 3a862c1..378c7ec 100644 --- a/gallery_dl/extractor/deviantart.py +++ b/gallery_dl/extractor/deviantart.py @@ -687,7 +687,7 @@ x2="45.4107524%" y2="71.4898596%" id="app-root-3">\ for folder in folders: if match(folder["name"]): return folder - elif folder["has_subfolders"]: + elif folder.get("has_subfolders"): for subfolder in folder["subfolders"]: if match(subfolder["name"]): return subfolder @@ -695,7 +695,7 @@ x2="45.4107524%" y2="71.4898596%" id="app-root-3">\ for folder in folders: if folder["folderid"] == uuid: return folder - elif folder["has_subfolders"]: + elif folder.get("has_subfolders"): for subfolder in folder["subfolders"]: if subfolder["folderid"] == uuid: return subfolder diff --git a/gallery_dl/extractor/discord.py b/gallery_dl/extractor/discord.py index 6a5fcc9..ac21fec 100644 --- a/gallery_dl/extractor/discord.py +++ b/gallery_dl/extractor/discord.py @@ -49,7 +49,10 @@ class DiscordExtractor(Extractor): text_content.append(field.get("name", "")) text_content.append(field.get("value", "")) - text_content.append(embed.get("footer", {}).get("text", "")) + try: + text_content.append(embed["footer"]["text"]) + except Exception: + pass if message.get("poll"): text_content.append(message["poll"]["question"]["text"]) @@ -224,10 +227,12 @@ class DiscordExtractor(Extractor): return self.server_metadata def build_server_and_channels(self, server_id): - server = self.api.get_server(server_id) - self.parse_server(server) + self.parse_server(self.api.get_server(server_id)) - for channel in self.api.get_server_channels(server_id): + for channel in sorted( + self.api.get_server_channels(server_id), + key=lambda ch: ch["type"] != 4 + ): self.parse_channel(channel) @@ -353,7 +358,8 @@ class DiscordAPI(): "limit": MESSAGES_BATCH, "before": before }) - before = messages[-1]["id"] + if messages: + before = messages[-1]["id"] return messages return self._pagination(_method, MESSAGES_BATCH) diff --git a/gallery_dl/extractor/everia.py b/gallery_dl/extractor/everia.py index 94444ff..e41f6f6 100644 --- a/gallery_dl/extractor/everia.py +++ b/gallery_dl/extractor/everia.py @@ -52,7 +52,7 @@ class EveriaPostExtractor(EveriaExtractor): def items(self): url = self.root + self.groups[0] page = self.request(url).text - content = text.extr(page, 'itemprop="text">', "</div>") + content = text.extr(page, 'itemprop="text">', "<h3") urls = re.findall(r'img.*?src="([^"]+)', content) data = { diff --git a/gallery_dl/extractor/gelbooru.py b/gallery_dl/extractor/gelbooru.py index 37c776e..eb07739 100644 --- a/gallery_dl/extractor/gelbooru.py +++ b/gallery_dl/extractor/gelbooru.py @@ -114,11 +114,12 @@ class GelbooruBase(): md5 = post["md5"] path = "/images/{}/{}/{}.webm".format(md5[0:2], md5[2:4], md5) post["_fallback"] = GelbooruBase._video_fallback(path) - url = "https://img3.gelbooru.com" + path + url = "https://img4.gelbooru.com" + path return url @staticmethod def _video_fallback(path): + yield "https://img3.gelbooru.com" + path yield "https://img2.gelbooru.com" + path yield "https://img1.gelbooru.com" + path diff --git a/gallery_dl/extractor/hentai2read.py b/gallery_dl/extractor/hentai2read.py index 9ab1411..1317ce9 100644 --- a/gallery_dl/extractor/hentai2read.py +++ b/gallery_dl/extractor/hentai2read.py @@ -25,26 +25,30 @@ class Hentai2readChapterExtractor(Hentai2readBase, ChapterExtractor): pattern = r"(?:https?://)?(?:www\.)?hentai2read\.com(/[^/?#]+/([^/?#]+))" example = "https://hentai2read.com/TITLE/1/" - def __init__(self, match): - self.chapter = match.group(2) - ChapterExtractor.__init__(self, match) - def metadata(self, page): title, pos = text.extract(page, "<title>", "</title>") manga_id, pos = text.extract(page, 'data-mid="', '"', pos) chapter_id, pos = text.extract(page, 'data-cid="', '"', pos) - chapter, sep, minor = self.chapter.partition(".") - match = re.match(r"Reading (.+) \(([^)]+)\) Hentai(?: by (.+))? - " + chapter, sep, minor = self.groups[1].partition(".") + + match = re.match(r"Reading (.+) \(([^)]+)\) Hentai(?: by (.*))? - " r"([^:]+): (.+) . Page 1 ", title) + if match: + manga, type, author, _, title = match.groups() + else: + self.log.warning("Failed to extract 'manga', 'type', 'author', " + "and 'title' metadata") + manga = type = author = title = "" + return { - "manga": match.group(1), + "manga": manga, "manga_id": text.parse_int(manga_id), "chapter": text.parse_int(chapter), "chapter_minor": sep + minor, "chapter_id": text.parse_int(chapter_id), - "type": match.group(2), - "author": match.group(3), - "title": match.group(5), + "type": type, + "author": author, + "title": title, "lang": "en", "language": "English", } diff --git a/gallery_dl/extractor/instagram.py b/gallery_dl/extractor/instagram.py index aa26408..432a7ad 100644 --- a/gallery_dl/extractor/instagram.py +++ b/gallery_dl/extractor/instagram.py @@ -29,6 +29,7 @@ class InstagramExtractor(Extractor): root = "https://www.instagram.com" cookies_domain = ".instagram.com" cookies_names = ("sessionid",) + useragent = util.USERAGENT_CHROME request_interval = (6.0, 12.0) def __init__(self, match): diff --git a/gallery_dl/extractor/issuu.py b/gallery_dl/extractor/issuu.py index 65717b4..abbdfd5 100644 --- a/gallery_dl/extractor/issuu.py +++ b/gallery_dl/extractor/issuu.py @@ -29,9 +29,11 @@ class IssuuPublicationExtractor(IssuuBase, GalleryExtractor): example = "https://issuu.com/issuu/docs/TITLE/" def metadata(self, page): - pos = page.rindex('id="initial-data"') - data = util.json_loads(text.unescape(text.rextract( - page, '<script data-json="', '"', pos)[0])) + + data = text.extr( + page, '{\\"documentTextVersion\\":', ']\\n"])</script>') + data = util.json_loads(text.unescape( + '{"":' + data.replace('\\"', '"'))) doc = data["initialDocumentData"]["document"] doc["date"] = text.parse_datetime( @@ -39,7 +41,7 @@ class IssuuPublicationExtractor(IssuuBase, GalleryExtractor): self._cnt = text.parse_int(doc["pageCount"]) self._tpl = "https://{}/{}-{}/jpg/page_{{}}.jpg".format( - data["config"]["hosts"]["image"], + "image.isu.pub", # data["config"]["hosts"]["image"], doc["revisionId"], doc["publicationId"], ) @@ -66,9 +68,8 @@ class IssuuUserExtractor(IssuuBase, Extractor): url = base + "/" + str(pnum) if pnum > 1 else base try: html = self.request(url).text - data = util.json_loads(text.unescape(text.extr( - html, '</main></div><script data-json="', '" id="'))) - docs = data["docs"] + data = text.extr(html, '\\"docs\\":', '}]\\n"]') + docs = util.json_loads(data.replace('\\"', '"')) except Exception as exc: self.log.debug("", exc_info=exc) return diff --git a/gallery_dl/extractor/kemonoparty.py b/gallery_dl/extractor/kemonoparty.py index 860e771..de7d040 100644 --- a/gallery_dl/extractor/kemonoparty.py +++ b/gallery_dl/extractor/kemonoparty.py @@ -123,6 +123,9 @@ class KemonopartyExtractor(Extractor): g(post) for g in generators): url = file["path"] + if "\\" in url: + file["path"] = url = url.replace("\\", "/") + match = find_hash(url) if match: file["hash"] = hash = match.group(1) diff --git a/gallery_dl/extractor/pixiv.py b/gallery_dl/extractor/pixiv.py index 8a4905d..e8050b3 100644 --- a/gallery_dl/extractor/pixiv.py +++ b/gallery_dl/extractor/pixiv.py @@ -15,7 +15,7 @@ from datetime import datetime, timedelta import itertools import hashlib -BASE_PATTERN = r"(?:https?://)?(?:www\.|touch\.)?pixiv\.net" +BASE_PATTERN = r"(?:https?://)?(?:www\.|touch\.)?ph?ixiv\.net" USER_PATTERN = BASE_PATTERN + r"/(?:en/)?users/(\d+)" @@ -531,7 +531,7 @@ class PixivMeExtractor(PixivExtractor): class PixivWorkExtractor(PixivExtractor): """Extractor for a single pixiv work/illustration""" subcategory = "work" - pattern = (r"(?:https?://)?(?:(?:www\.|touch\.)?pixiv\.net" + pattern = (r"(?:https?://)?(?:(?:www\.|touch\.)?ph?ixiv\.net" r"/(?:(?:en/)?artworks/" r"|member_illust\.php\?(?:[^&]+&)*illust_id=)(\d+)" r"|(?:i(?:\d+\.pixiv|\.pximg)\.net" diff --git a/gallery_dl/extractor/readcomiconline.py b/gallery_dl/extractor/readcomiconline.py index c0374eb..2f2daca 100644 --- a/gallery_dl/extractor/readcomiconline.py +++ b/gallery_dl/extractor/readcomiconline.py @@ -85,7 +85,7 @@ class ReadcomiconlineIssueExtractor(ReadcomiconlineBase, ChapterExtractor): replacements = re.findall( r"l = l\.replace\(/([^/]+)/g, [\"']([^\"']*)", page) - for block in page.split(" pth = '")[1:]: + for block in page.split("\t\tpht = '")[1:]: pth = text.extr(block, "", "'") for needle, repl in re.findall( @@ -129,7 +129,7 @@ class ReadcomiconlineComicExtractor(ReadcomiconlineBase, MangaExtractor): def baeu(url, root="", root_blogspot="https://2.bp.blogspot.com"): - """https://readcomiconline.li/Scripts/rguard.min.js""" + """https://readcomiconline.li/Scripts/rguard.min.js?v=1.5.4""" if not root: root = root_blogspot diff --git a/gallery_dl/extractor/rule34xyz.py b/gallery_dl/extractor/rule34xyz.py index 3b8d344..411a71a 100644 --- a/gallery_dl/extractor/rule34xyz.py +++ b/gallery_dl/extractor/rule34xyz.py @@ -23,10 +23,18 @@ class Rule34xyzExtractor(BooruExtractor): per_page = 60 TAG_TYPES = { - 0: "general", - 1: "copyright", - 2: "character", - 3: "artist", + None: "general", + 0 : "general", + 1 : "general", + 2 : "copyright", + 4 : "character", + 8 : "artist", + } + FORMATS = { + "10" : "pic.jpg", + "100": "mov.mp4", + "101": "mov720.mp4", + "102": "mov480.mp4", } def _init(self): @@ -36,49 +44,49 @@ class Rule34xyzExtractor(BooruExtractor): formats = formats.split(",") self.formats = formats else: - self.formats = ("10", "40", "41", "2") + self.formats = ("100", "101", "102", "10") def _file_url(self, post): - post["files"] = files = { - str(link["type"]): link["url"] - for link in post.pop("imageLinks") - } + files = post["files"] for fmt in self.formats: if fmt in files: + extension = self.FORMATS.get(fmt) break else: - fmt = "2" self.log.warning("%s: Requested format not available", post["id"]) + fmt = next(iter(files)) - post["file_url"] = url = files[fmt] + post_id = post["id"] + root = self.root_cdn if files[fmt][0] else self.root + post["file_url"] = url = "{}/posts/{}/{}/{}.{}".format( + root, post_id // 1000, post_id, post_id, extension) post["format_id"] = fmt - post["format"] = url.rsplit(".", 2)[1] + post["format"] = extension.partition(".")[0] + return url def _prepare(self, post): - post.pop("filesPreview", None) - post.pop("tagsWithType", None) + post.pop("files", None) post["date"] = text.parse_datetime( - post["created"][:19], "%Y-%m-%dT%H:%M:%S") + post["created"], "%Y-%m-%dT%H:%M:%S.%fZ") + post["filename"], _, post["format"] = post["filename"].rpartition(".") + if "tags" in post: + post["tags"] = [t["value"] for t in post["tags"]] def _tags(self, post, _): - if post.get("tagsWithType") is None: + if "tags" not in post: post.update(self._fetch_post(post["id"])) tags = collections.defaultdict(list) - tagslist = [] - for tag in post["tagsWithType"]: - value = tag["value"] - tagslist.append(value) - tags[tag["type"]].append(value) + for tag in post["tags"]: + tags[tag["type"]].append(tag["value"]) types = self.TAG_TYPES for type, values in tags.items(): post["tags_" + types[type]] = values - post["tags"] = tagslist def _fetch_post(self, post_id): - url = "{}/api/post/{}".format(self.root, post_id) + url = "{}/api/v2/post/{}".format(self.root, post_id) return self.request(url).json() def _pagination(self, endpoint, params=None): @@ -86,22 +94,22 @@ class Rule34xyzExtractor(BooruExtractor): if params is None: params = {} - params["IncludeLinks"] = "true" - params["IncludeTags"] = "true" - params["OrderBy"] = "0" params["Skip"] = self.page_start * self.per_page - params["Take"] = self.per_page - params["DisableTotal"] = "true" + params["take"] = self.per_page + params["CountTotal"] = False + params["IncludeLinks"] = True + params["OrderBy"] = 0 threshold = self.per_page while True: - data = self.request(url, params=params).json() + data = self.request(url, method="POST", json=params).json() yield from data["items"] if len(data["items"]) < threshold: return - params["Skip"] += params["Take"] + params["Skip"] += self.per_page + params["cursor"] = data["cursor"] class Rule34xyzPostExtractor(Rule34xyzExtractor): @@ -125,9 +133,8 @@ class Rule34xyzPlaylistExtractor(Rule34xyzExtractor): return {"playlist_id": self.groups[0]} def posts(self): - endpoint = "/playlist-item" - params = {"PlaylistId": self.groups[0]} - return self._pagination(endpoint, params) + endpoint = "/v2/post/search/playlist/" + self.groups[0] + return self._pagination(endpoint) class Rule34xyzTagExtractor(Rule34xyzExtractor): @@ -138,10 +145,11 @@ class Rule34xyzTagExtractor(Rule34xyzExtractor): example = "https://rule34.xyz/TAG" def metadata(self): - self.tags = text.unquote(self.groups[0]).replace("_", " ") - return {"search_tags": self.tags} + self.tags = text.unquote(text.unquote( + self.groups[0]).replace("_", " ")).split("|") + return {"search_tags": ", ".join(self.tags)} def posts(self): - endpoint = "/post/search" - params = {"Tag": self.tags} + endpoint = "/v2/post/search/root" + params = {"includeTags": self.tags} return self._pagination(endpoint, params) diff --git a/gallery_dl/extractor/tumblr.py b/gallery_dl/extractor/tumblr.py index 8d1fcde..6f2114e 100644 --- a/gallery_dl/extractor/tumblr.py +++ b/gallery_dl/extractor/tumblr.py @@ -17,7 +17,7 @@ import re BASE_PATTERN = ( r"(?:tumblr:(?:https?://)?([^/]+)|" r"(?:https?://)?" - r"(?:www\.tumblr\.com/(?:blog/(?:view/)?)?([\w-]+)|" + r"(?:(?:www\.)?tumblr\.com/(?:blog/(?:view/)?)?([\w-]+)|" r"([\w-]+\.tumblr\.com)))" ) @@ -357,7 +357,7 @@ class TumblrLikesExtractor(TumblrExtractor): class TumblrSearchExtractor(TumblrExtractor): """Extractor for a Tumblr search""" subcategory = "search" - pattern = (BASE_PATTERN + r"/search/([^/?#]+)" + pattern = (r"(?:https?://)?(?:www\.)?tumblr\.com/search/([^/?#]+)" r"(?:/([^/?#]+)(?:/([^/?#]+))?)?(?:/?\?([^#]+))?") example = "https://www.tumblr.com/search/QUERY" diff --git a/gallery_dl/extractor/webtoons.py b/gallery_dl/extractor/webtoons.py index 008ae6e..8ff32af 100644 --- a/gallery_dl/extractor/webtoons.py +++ b/gallery_dl/extractor/webtoons.py @@ -12,13 +12,15 @@ from .common import GalleryExtractor, Extractor, Message from .. import exception, text, util -BASE_PATTERN = r"(?:https?://)?(?:www\.)?webtoons\.com/(([^/?#]+)" +BASE_PATTERN = r"(?:https?://)?(?:www\.)?webtoons\.com" +LANG_PATTERN = BASE_PATTERN + r"/(([^/?#]+)" class WebtoonsBase(): category = "webtoons" root = "https://www.webtoons.com" cookies_domain = ".webtoons.com" + request_interval = (0.5, 1.5) def setup_agegate_cookies(self): self.cookies_update({ @@ -34,7 +36,7 @@ class WebtoonsBase(): response = Extractor.request(self, url, **kwargs) if response.history and "/ageGate" in response.url: raise exception.StopExtraction( - "HTTP redirect to age gate check ('%s')", response.request.url) + "HTTP redirect to age gate check ('%s')", response.url) return response @@ -44,47 +46,19 @@ class WebtoonsEpisodeExtractor(WebtoonsBase, GalleryExtractor): directory_fmt = ("{category}", "{comic}") filename_fmt = "{episode_no}-{num:>02}.{extension}" archive_fmt = "{title_no}_{episode_no}_{num}" - pattern = (BASE_PATTERN + r"/([^/?#]+)/([^/?#]+)/(?:[^/?#]+))" - r"/viewer(?:\?([^#'\"]+))") + pattern = (LANG_PATTERN + r"/([^/?#]+)/([^/?#]+)/[^/?#]+)" + r"/viewer\?([^#'\"]+)") example = ("https://www.webtoons.com/en/GENRE/TITLE/NAME/viewer" "?title_no=123&episode_no=12345") - test = ( - (("https://www.webtoons.com/en/comedy/safely-endangered" - "/ep-572-earth/viewer?title_no=352&episode_no=572"), { - "url": "55bec5d7c42aba19e3d0d56db25fdf0b0b13be38", - "content": ("1748c7e82b6db910fa179f6dc7c4281b0f680fa7", - "42055e44659f6ffc410b3fb6557346dfbb993df3", - "49e1f2def04c6f7a6a3dacf245a1cd9abe77a6a9"), - "count": 5, - }), - (("https://www.webtoons.com/en/challenge/punderworld" - "/happy-earth-day-/viewer?title_no=312584&episode_no=40"), { - "exception": exception.NotFoundError, - "keyword": { - "comic": "punderworld", - "description": str, - "episode": "36", - "episode_no": "40", - "genre": "challenge", - "title": r"re:^Punderworld - .+", - "title_no": "312584", - }, - }), - ) - - def __init__(self, match): - self.path, self.lang, self.genre, self.comic, self.query = \ - match.groups() - - url = "{}/{}/viewer?{}".format(self.root, self.path, self.query) - GalleryExtractor.__init__(self, match, url) def _init(self): self.setup_agegate_cookies() - params = text.parse_query(self.query) + path, self.lang, self.genre, self.comic, query = self.groups + params = text.parse_query(query) self.title_no = params.get("title_no") self.episode_no = params.get("episode_no") + self.gallery_url = "{}/{}/viewer?{}".format(self.root, path, query) def metadata(self, page): extr = text.extract_from(page) @@ -124,32 +98,49 @@ class WebtoonsEpisodeExtractor(WebtoonsBase, GalleryExtractor): "language" : util.code_to_language(self.lang), } - @staticmethod - def images(page): - return [ - (url.replace("://webtoon-phinf.", "://swebtoon-phinf."), None) - for url in text.extract_iter( - page, 'class="_images" data-url="', '"') - ] + def images(self, page): + quality = self.config("quality") + if quality is None or quality == "original": + quality = {"jpg": False, "jpeg": False, "webp": False} + elif not quality: + quality = None + elif isinstance(quality, str): + quality = {"jpg": quality, "jpeg": quality} + elif isinstance(quality, int): + quality = "q" + str(quality) + quality = {"jpg": quality, "jpeg": quality} + elif not isinstance(quality, dict): + quality = None + + results = [] + for url in text.extract_iter( + page, 'class="_images" data-url="', '"'): + + if quality is not None: + path, _, query = url.rpartition("?") + type = quality.get(path.rpartition(".")[2].lower()) + if type is False: + url = path + elif type: + url = "{}?type={}".format(path, type) + + url = url.replace("://webtoon-phinf.", "://swebtoon-phinf.") + results.append((url, None)) + return results class WebtoonsComicExtractor(WebtoonsBase, Extractor): """Extractor for an entire comic on webtoons.com""" subcategory = "comic" categorytransfer = True - pattern = (BASE_PATTERN + r"/([^/?#]+)/([^/?#]+))" - r"/list(?:\?([^#]+))") + pattern = LANG_PATTERN + r"/([^/?#]+)/([^/?#]+))/list\?([^#]+)" example = "https://www.webtoons.com/en/GENRE/TITLE/list?title_no=123" - def __init__(self, match): - Extractor.__init__(self, match) - self.path, self.lang, self.genre, self.comic, self.query = \ - match.groups() - def _init(self): self.setup_agegate_cookies() - params = text.parse_query(self.query) + self.path, self.lang, self.genre, self.comic, query = self.groups + params = text.parse_query(query) self.title_no = params.get("title_no") self.page_no = text.parse_int(params.get("page"), 1) @@ -164,7 +155,7 @@ class WebtoonsComicExtractor(WebtoonsBase, Extractor): path = "/{}/list?title_no={}&page={}".format( self.path, self.title_no, self.page_no) - if page and path not in page: + if page is not None and path not in page: return response = self.request(self.root + path) @@ -182,11 +173,47 @@ class WebtoonsComicExtractor(WebtoonsBase, Extractor): self.page_no += 1 - @staticmethod - def get_episode_urls(page): + def get_episode_urls(self, page): """Extract and return all episode urls in 'page'""" page = text.extr(page, 'id="_listUl"', '</ul>') return [ match.group(0) for match in WebtoonsEpisodeExtractor.pattern.finditer(page) ] + + +class WebtoonsArtistExtractor(WebtoonsBase, Extractor): + """Extractor for webtoons.com artists""" + subcategory = "artist" + pattern = BASE_PATTERN + r"/p/community/([^/?#]+)/u/([^/?#]+)" + example = "https://www.webtoons.com/p/community/LANG/u/ARTIST" + + def items(self): + self.setup_agegate_cookies() + + for comic in self.comics(): + comic["_extractor"] = WebtoonsComicExtractor + comic_url = self.root + comic["extra"]["episodeListPath"] + yield Message.Queue, comic_url, comic + + def comics(self): + lang, artist = self.groups + language = util.code_to_language(lang).upper() + + url = "{}/p/community/{}/u/{}".format( + self.root, lang, artist) + page = self.request(url).text + creator_id = text.extr(page, '\\"creatorId\\":\\"', '\\') + + url = "{}/p/community/api/v1/creator/{}/titles".format( + self.root, creator_id) + params = { + "language": language, + "nextSize": "50", + } + headers = { + "language": language, + } + data = self.request(url, params=params, headers=headers).json() + + return data["result"]["titles"] diff --git a/gallery_dl/extractor/zerochan.py b/gallery_dl/extractor/zerochan.py index ac1400e..0ad73c0 100644 --- a/gallery_dl/extractor/zerochan.py +++ b/gallery_dl/extractor/zerochan.py @@ -74,7 +74,6 @@ class ZerochanExtractor(BooruExtractor): extr = text.extract_from(page) data = { "id" : text.parse_int(entry_id), - "author" : jsonld["author"]["name"], "file_url": jsonld["contentUrl"], "date" : text.parse_datetime(jsonld["datePublished"]), "width" : text.parse_int(jsonld["width"][:-3]), @@ -88,6 +87,11 @@ class ZerochanExtractor(BooruExtractor): 'id="source-url"', '</p>').rpartition("</s>")[2])), } + try: + data["author"] = jsonld["author"]["name"] + except Exception: + data["author"] = "" + html = data["tags"] tags = data["tags"] = [] for tag in html.split("<li class=")[1:]: diff --git a/gallery_dl/extractor/zzup.py b/gallery_dl/extractor/zzup.py index 05b12b4..20454b4 100644 --- a/gallery_dl/extractor/zzup.py +++ b/gallery_dl/extractor/zzup.py @@ -16,7 +16,7 @@ class ZzupGalleryExtractor(GalleryExtractor): filename_fmt = "{num:>03}.{extension}" archive_fmt = "{slug}_{num}" root = "https://zzup.com" - pattern = (r"(?:https?://)?(up\.|www\.)?zzup\.com(/(?:viewalbum|content)" + pattern = (r"(?:https?://)?(up\.|w+\.)?zzup\.com(/(?:viewalbum|content)" r"/[\w=]+/([^/?#]+)/[\w=]+)/(?:index|page-\d+)\.html") example = "https://zzup.com/content/xyz=/12345_TITLE/123=/index.html" diff --git a/gallery_dl/formatter.py b/gallery_dl/formatter.py index e662c34..6affc3e 100644 --- a/gallery_dl/formatter.py +++ b/gallery_dl/formatter.py @@ -495,6 +495,8 @@ _CONVERSIONS = { "s": str, "r": repr, "a": ascii, + "i": int, + "f": float, } _FORMAT_SPECIFIERS = { "?": _parse_optional, diff --git a/gallery_dl/path.py b/gallery_dl/path.py index 21e1aa0..54cf126 100644 --- a/gallery_dl/path.py +++ b/gallery_dl/path.py @@ -269,7 +269,7 @@ class PathFormat(): try: for fmt in self.directory_formatters: segment = fmt(kwdict).strip() - if strip and segment != "..": + if strip and segment not in {".", ".."}: # remove trailing dots and spaces (#647) segment = segment.rstrip(strip) if segment: diff --git a/gallery_dl/postprocessor/metadata.py b/gallery_dl/postprocessor/metadata.py index 3ef9fbc..fbb3fb8 100644 --- a/gallery_dl/postprocessor/metadata.py +++ b/gallery_dl/postprocessor/metadata.py @@ -108,6 +108,7 @@ class MetadataPP(PostProcessor): self.omode = options.get("open", omode) self.encoding = options.get("encoding", "utf-8") self.skip = options.get("skip", False) + self.meta_path = options.get("metadata-path") def run(self, pathfmt): archive = self.archive @@ -120,6 +121,9 @@ class MetadataPP(PostProcessor): directory = self._directory(pathfmt) path = directory + self._filename(pathfmt) + if self.meta_path is not None: + pathfmt.kwdict[self.meta_path] = path + if self.skip and os.path.exists(path): return @@ -180,7 +184,10 @@ class MetadataPP(PostProcessor): pathfmt.directory_formatters = self._directory_formatters pathfmt.directory_conditions = () segments = pathfmt.build_directory(pathfmt.kwdict) - directory = pathfmt.clean_path(os.sep.join(segments) + os.sep) + if segments: + directory = pathfmt.clean_path(os.sep.join(segments) + os.sep) + else: + directory = "." + os.sep return os.path.join(self._base(pathfmt), directory) finally: pathfmt.directory_conditions = conditions diff --git a/gallery_dl/postprocessor/ugoira.py b/gallery_dl/postprocessor/ugoira.py index 3a32b39..c1bfc20 100644 --- a/gallery_dl/postprocessor/ugoira.py +++ b/gallery_dl/postprocessor/ugoira.py @@ -156,12 +156,7 @@ class UgoiraPP(PostProcessor): return self.log.debug("", exc_info=exc) if self.convert(pathfmt, tempdir): - if self.delete: - pathfmt.delete = True - elif pathfmt.extension != "zip": - self.log.info(pathfmt.filename) - pathfmt.set_extension("zip") - pathfmt.build_path() + pathfmt.delete = self.delete def convert_from_files(self, pathfmt): if not self._convert_files: diff --git a/gallery_dl/util.py b/gallery_dl/util.py index 76e6517..eabd4ab 100644 --- a/gallery_dl/util.py +++ b/gallery_dl/util.py @@ -700,6 +700,9 @@ EXECUTABLE = getattr(sys, "frozen", False) USERAGENT = "gallery-dl/" + version.__version__ USERAGENT_FIREFOX = ("Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:{}.0) " "Gecko/20100101 Firefox/{}.0").format(_ff_ver, _ff_ver) +USERAGENT_CHROME = ("Mozilla/5.0 (Windows NT 10.0; Win64; x64) " + "AppleWebKit/537.36 (KHTML, like Gecko) Chrome/134.0.0.0 " + "Safari/537.36") SPECIAL_EXTRACTORS = {"oauth", "recursive", "generic"} GLOBALS = { "contains" : contains, diff --git a/gallery_dl/version.py b/gallery_dl/version.py index 43b234d..87169e2 100644 --- a/gallery_dl/version.py +++ b/gallery_dl/version.py @@ -6,5 +6,5 @@ # it under the terms of the GNU General Public License version 2 as # published by the Free Software Foundation. -__version__ = "1.29.3" +__version__ = "1.29.4" __variant__ = None diff --git a/test/test_formatter.py b/test/test_formatter.py index c0b504d..646f179 100644 --- a/test/test_formatter.py +++ b/test/test_formatter.py @@ -35,6 +35,8 @@ class TestFormatter(unittest.TestCase): "ds": "2010-01-01T01:00:00+0100", "dt": datetime.datetime(2010, 1, 1), "dt_dst": datetime.datetime(2010, 6, 1), + "i_str": "12345", + "f_str": "12.45", "name": "Name", "title1": "Title", "title2": "", @@ -70,6 +72,9 @@ class TestFormatter(unittest.TestCase): self._run_test("{a!L}", 11) self._run_test("{l!L}", 3) self._run_test("{d!L}", 3) + self._run_test("{i_str!i}", 12345) + self._run_test("{i_str!f}", 12345.0) + self._run_test("{f_str!f}", 12.45) with self.assertRaises(KeyError): self._run_test("{a!q}", "hello world") @@ -483,10 +488,10 @@ def noarg(): fmt4 = formatter.parse("\fM " + path + ":lengths") self.assertEqual(fmt1.format_map(self.kwdict), "'Title' by Name") - self.assertEqual(fmt2.format_map(self.kwdict), "126") + self.assertEqual(fmt2.format_map(self.kwdict), "136") self.assertEqual(fmt3.format_map(self.kwdict), "'Title' by Name") - self.assertEqual(fmt4.format_map(self.kwdict), "126") + self.assertEqual(fmt4.format_map(self.kwdict), "136") with self.assertRaises(TypeError): self.assertEqual(fmt0.format_map(self.kwdict), "") diff --git a/test/test_postprocessor.py b/test/test_postprocessor.py index 2941b81..8b073b4 100644 --- a/test/test_postprocessor.py +++ b/test/test_postprocessor.py @@ -511,6 +511,17 @@ class MetadataTest(BasePostprocessorTest): path = self.pathfmt.realdirectory + "../json/12500/file.ext.json" m.assert_called_once_with(path, "w", encoding="utf-8") + def test_metadata_directory_empty(self): + self._create( + {"directory": []}, + ) + + with patch("builtins.open", mock_open()) as m: + self._trigger() + + path = self.pathfmt.realdirectory + "./file.ext.json" + m.assert_called_once_with(path, "w", encoding="utf-8") + def test_metadata_basedirectory(self): self._create({"base-directory": True}) @@ -544,6 +555,16 @@ class MetadataTest(BasePostprocessorTest): path = self.pathfmt.realdirectory + "test_file__meta_.data" m.assert_called_once_with(path, "w", encoding="utf-8") + def test_metadata_meta_path(self): + self._create({ + "metadata-path": "_meta_path", + }) + + self._trigger() + + self.assertEqual(self.pathfmt.kwdict["_meta_path"], + self.pathfmt.realpath + ".json") + def test_metadata_stdout(self): self._create({"filename": "-", "indent": None, "sort": True}) |