New upstream version 1.10.2upstream/1.10.2

3 files changed, 61 insertions, 16 deletions

diff --git a/docs/configuration.rst b/docs/configuration.rst
index c6f757d..0e2e355 100644
--- a/docs/configuration.rst
+++ b/docs/configuration.rst
@@ -108,21 +108,36 @@ Description Directory path used as the base for all download destinations.
 =========== =====
 
 
-extractor.*.restrict-filenames
-------------------------------
+extractor.*.path-restrict
+-------------------------
 =========== =====
 Type        ``string``
 Default     ``"auto"``
-Example     ``"/!? ()[]{}"``
-Description Characters to replace with underscores (``_``) when generating
-            directory and file names.
+Example     ``"/!? (){}"``
+Description Set of characters to replace with underscores (``_``)
+            in generated path segment names.
 
             Special values:
 
             * ``"auto"``: Use characters from ``"unix"`` or ``"windows"``
               depending on the local operating system
             * ``"unix"``: ``"/"``
-            * ``"windows"``: ``"<>:\"\\|/?*"``
+            * ``"windows"``: ``"\\\\|/<>:\"?*"``
+
+            Note: In a set with 2 or more characters, ``[]^-\`` need to be
+            escaped with backslashes, e.g. ``"\\[\\]"``
+=========== =====
+
+
+extractor.*.path-remove
+-----------------------
+=========== =====
+Type        ``string``
+Default     ``"\\u0000-\\u001f\\u007f"`` (ASCII control characters)
+Description Set of characters to remove from generated path names.
+
+            Note: In a set with 2 or more characters, ``[]^-\`` need to be
+            escaped with backslashes, e.g. ``"\\[\\]"``
 =========== =====
 
 
@@ -131,8 +146,11 @@ extractor.*.skip
 =========== =====
 Type        ``bool`` or ``string``
 Default     ``true``
-Description Controls the behavior when downloading files whose filename
-            already exists.
+Description Controls the behavior when downloading files that have been
+            downloaded before, i.e. a file with the same filename already
+            exists or its ID is in a `download archive`__.
+
+            __ `extractor.*.archive`_
 
             * ``true``: Skip downloads
             * ``false``: Overwrite already existing files
@@ -144,6 +162,9 @@ Description Controls the behavior when downloading files whose filename
             * ``"exit"``: Exit the program altogether
             * ``"exit:N"``: Skip downloads and exit the program
               after ``N`` consecutive skips
+
+            * ``"enumerate"``: Append a numeric suffix to the end of the
+              original filename (``file.ext.1``, ``file.ext.2``, etc)
 =========== =====
 
 
@@ -555,6 +576,15 @@ Description Download original files if available.
 =========== =====
 
 
+extractor.deviantart.quality
+----------------------------
+=========== =====
+Type        ``integer``
+Default     ``100``
+Description JPEG compression quality for newer images hosted on wixmp servers.
+=========== =====
+
+
 extractor.deviantart.refresh-token
 ----------------------------------
 =========== =====
@@ -1098,6 +1128,16 @@ Description Certificate validation during file downloads.
 =========== =====
 
 
+downloader.http.adjust-extensions
+---------------------------------
+=========== =====
+Type        ``bool``
+Default     ``true``
+Description Check the file headers of ``jpg``, ``png``, and ``gif`` files
+            and adjust their filename extensions if they do not match.
+=========== =====
+
+
 downloader.ytdl.format
 ----------------------
 =========== =====
@@ -1772,7 +1812,7 @@ Description An object with the ``name`` of a post-processor and its options.
 .. _timeout:           https://docs.python-requests.org/en/latest/user/advanced/#timeouts
 .. _verify:            https://docs.python-requests.org/en/master/user/advanced/#ssl-cert-verification
 .. _Last-Modified:     https://www.w3.org/Protocols/rfc2616/rfc2616-sec14.html#sec14.29
-.. _`Requests' proxy documentation`: http://docs.python-requests.org/en/master/user/advanced/#proxies
+.. _`Requests' proxy documentation`: https://2.python-requests.org/en/master/user/advanced/#proxies
 .. _format string:     https://docs.python.org/3/library/string.html#formatstrings
 .. _format strings:    https://docs.python.org/3/library/string.html#formatstrings
 .. _strptime:          https://docs.python.org/3/library/datetime.html#strftime-strptime-behavior
@@ -1780,5 +1820,5 @@ Description An object with the ``name`` of a post-processor and its options.
 .. _webbrowser.open(): https://docs.python.org/3/library/webbrowser.html
 .. _datetime:          https://docs.python.org/3/library/datetime.html#datetime-objects
 .. _datetime.max:      https://docs.python.org/3/library/datetime.html#datetime.datetime.max
-.. _Authentication:    https://github.com/mikf/gallery-dl#5authentication
+.. _Authentication:    https://github.com/mikf/gallery-dl#authentication
 .. _youtube-dl:        https://github.com/ytdl-org/youtube-dl
diff --git a/docs/gallery-dl.conf b/docs/gallery-dl.conf
index a4a9ee0..b9ff32d 100644
--- a/docs/gallery-dl.conf
+++ b/docs/gallery-dl.conf
@@ -8,8 +8,9 @@
         "proxy": null,
         "skip": true,
         "sleep": 0,
+        "path-restrict": "auto",
+        "path-remove": "\\u0000-\\u001f\\u007f",
         "user-agent": "Mozilla/5.0 (X11; Linux x86_64; rv:68.0) Gecko/20100101 Firefox/68.0",
-        "restrict-filenames": "auto",
 
         "artstation":
         {
@@ -30,6 +31,7 @@
             "mature": true,
             "metadata": false,
             "original": true,
+            "quality": 100,
             "wait-min": 0
         },
         "exhentai":
@@ -154,6 +156,7 @@
 
         "http":
         {
+            "adjust-extensions": true,
             "mtime": true,
             "rate": null,
             "retries": 4,
diff --git a/docs/supportedsites.rst b/docs/supportedsites.rst
index d2fb4ea..05c8555 100644
--- a/docs/supportedsites.rst
+++ b/docs/supportedsites.rst
@@ -28,6 +28,7 @@ Dynasty Reader       https://dynasty-scans.com/          Chapters, individual Im
 E-Hentai             https://e-hentai.org/               Favorites, Galleries, Search Results               Optional
 e621                 https://e621.net/                   Pools, Popular Images, Posts, Tag-Searches
 EroLord.com          http://erolord.com/                 Galleries
+ExHentai             https://exhentai.org/               Favorites, Galleries, Search Results               Optional
 Fallen Angels Scans  https://www.fascans.com/            Chapters, Manga
 Fashion Nova         https://www.fashionnova.com/        Collections, Products
 Fireden              https://boards.fireden.net/         Threads
@@ -47,11 +48,11 @@ Hypnohub             https://hypnohub.net/               Pools, Popular Images,
 Idol Complex         https://idol.sankakucomplex.com/    Pools, Posts, Tag-Searches                         Optional
 ImageBam             http://www.imagebam.com/            Galleries, individual Images
 ImageFap             https://imagefap.com/               Images from Users, Galleries, individual Images
-ImgBB                https://imgbb.com/                  Images from Users, Albums                          Optional
+ImgBB                https://imgbb.com/                  Images from Users, Albums, individual Images       Optional
 imgbox               https://imgbox.com/                 Galleries, individual Images
 imgth                https://imgth.com/                  Galleries
-imgur                https://imgur.com/                  Albums, individual Images
-Instagram            https://www.instagram.com/          Images from Users, individual Images, Tag-Searches Optional
+imgur                https://imgur.com/                  Albums, Galleries, individual Images
+Instagram            https://www.instagram.com/          |instagram-C|                                      Optional
 Jaimini's Box        https://jaiminisbox.com/reader/     Chapters, Manga
 Joyreactor           http://joyreactor.cc/               |joyreactor-C|
 Keenspot             http://www.keenspot.com/            Comics
@@ -77,7 +78,7 @@ Niconico Seiga       https://seiga.nicovideo.jp/         Images from Users, indi
 nijie                https://nijie.info/                 |nijie-C|                                          Required
 NSFWalbum.com        https://nsfwalbum.com/              Albums
 Nyafuu Archive       https://archive.nyafuu.org/         Threads
-Patreon              https://www.patreon.com/            Images from Users, Creators
+Patreon              https://www.patreon.com/            Images from Users, Creators, Posts
 Pawoo                https://pawoo.net/                  Images from Users, Images from Statuses
 Photobucket          https://photobucket.com/            Albums, individual Images
 Piczel               https://piczel.tv/                  Images from Users, Folders, individual Images
@@ -100,7 +101,7 @@ Sankaku Complex      https://www.sankakucomplex.com/     Articles, Tag-Searches
 Sen Manga            https://raw.senmanga.com/           Chapters
 Sense-Scans          http://sensescans.com/reader/       Chapters, Manga
 Sex.com              https://www.sex.com/                Boards, Pins, related Pins, Search Results
-Simply Hentai        https://www.simply-hentai.com/      Galleries, individual Images, Videos
+Simply Hentai        https://www.simply-hentai.com/      Galleries
 SlickPic             https://www.slickpic.com/           Images from Users, Albums
 SlideShare           https://www.slideshare.net/         Presentations
 SmugMug              https://www.smugmug.com/            |smugmug-C|                                        Optional (OAuth)
@@ -134,6 +135,7 @@ Turboimagehost       https://www.turboimagehost.com/     individual Images
 .. |deviantart-C| replace:: Collections, Deviations, Favorites, Folders, Galleries, Journals, Popular Images, Scraps, Sta.sh
 .. |flickr-C| replace:: Images from Users, Albums, Favorites, Galleries, Groups, individual Images, Search Results
 .. |hentaifoundry-C| replace:: Images from Users, Favorites, individual Images, Popular Images, Recent Images, Scraps
+.. |instagram-C| replace:: Images from Users, Channels, individual Images, Stories, Tag-Searches
 .. |joyreactor-C| replace:: Images from Users, Posts, Search Results, Tag-Searches
 .. |nijie-C| replace:: Images from Users, Doujin, Favorites, individual Images
 .. |pixiv-C| replace:: Images from Users, Favorites, Follows, pixiv.me Links, Rankings, Search Results, Individual Images