[PATCH setup] Query the user if a corrupt local file is found

classic Classic list List threaded Threaded
5 messages Options
Reply | Threaded
Open this post in threaded view
|

[PATCH setup] Query the user if a corrupt local file is found

Ken Brown-6
If a corrupt file is found in one of the selected mirror site
directories, offer to delete it instead of making this a fatal error.
Do this only on the first call to check_for cached().  If the corrupt
file is still there on the second call, then the deletion failed, and
the user will have to fix the problem.

See https://cygwin.com/ml/cygwin/2017-12/msg00122.html for discussion.
---
 download.cc | 17 ++++++++++++-----
 download.h  |  4 +++-
 res.rc      |  3 ++-
 resource.h  |  1 +
 4 files changed, 18 insertions(+), 7 deletions(-)

diff --git a/download.cc b/download.cc
index b606eb1..697f94a 100644
--- a/download.cc
+++ b/download.cc
@@ -71,7 +71,7 @@ validateCachedPackage (const std::string& fullname, packagesource & pkgsource)
 /* 0 on failure
  */
 int
-check_for_cached (packagesource & pkgsource, bool mirror_mode)
+check_for_cached (packagesource & pkgsource, bool mirror_mode, bool query_corrupt, HWND owner)
 {
   // Already found one.
   if (pkgsource.Cached())
@@ -114,6 +114,12 @@ check_for_cached (packagesource & pkgsource, bool mirror_mode)
     {
       if (validateCachedPackage (fullname, pkgsource))
         pkgsource.set_cached (fullname);
+      else if (query_corrupt && owner && !unattended_mode
+       && yesno (owner, IDS_QUERY_CORRUPT, fullname.c_str()) == IDYES)
+ {
+  io_stream::remove (fullname);
+  continue;
+ }
       else
         throw new Exception (TOSTRING(__LINE__) " " __FILE__,
             "Package validation failure for " + fullname,
@@ -266,13 +272,14 @@ do_download_thread (HINSTANCE h, HWND owner)
     {
       if (pkg.picked())
  {
-    if (!check_for_cached (*version.source()))
-      total_download_bytes += version.source()->size;
+  if (!check_for_cached (*version.source(), false, true, owner))
+    total_download_bytes += version.source()->size;
  }
       if (pkg.srcpicked () || IncludeSource)
  {
-    if (!check_for_cached (*sourceversion.source()))
-      total_download_bytes += sourceversion.source()->size;
+  if (!check_for_cached (*sourceversion.source(), false,
+ true, owner))
+    total_download_bytes += sourceversion.source()->size;
  }
     }
   catch (Exception * e)
diff --git a/download.h b/download.h
index 117a44d..e9eb257 100644
--- a/download.h
+++ b/download.h
@@ -16,7 +16,9 @@
 #ifndef SETUP_DOWNLOAD_H
 #define SETUP_DOWNLOAD_H
 
+#include "win32.h"
+
 class packagesource;
-int check_for_cached (packagesource & pkgsource, bool = false);
+int check_for_cached (packagesource & pkgsource, bool mirror_mode = false, bool query_corrupt = false, HWND owner = NULL);
 
 #endif /* SETUP_DOWNLOAD_H */
diff --git a/res.rc b/res.rc
index 901cf76..31c8f20 100644
--- a/res.rc
+++ b/res.rc
@@ -550,7 +550,8 @@ BEGIN
     IDS_DOWNLOAD_INCOMPLETE_EXIT  "Download incomplete.  Check %s for details"
     IDS_INSTALL_ERROR    "Installation error (%s), Continue with other packages?"
     IDS_INSTALL_INCOMPLETE  "Installation incomplete.  Check %s for details"
-    IDS_CORRUPT_PACKAGE     "Package file %s has a corrupt local copy, please remove and retry."
+    IDS_CORRUPT_PACKAGE     "Package %s has a corrupt local copy, please remove and retry."
+    IDS_QUERY_CORRUPT       "The file %s is corrupt.  Delete it?."
     IDS_SKIP_PACKAGE    "%s\nDo you want to skip this package ?"
     IDS_UNCAUGHT_EXCEPTION  "Fatal Error: Uncaught Exception\nThread: %s\nType: %s\nMessage: %s"
     IDS_UNCAUGHT_EXCEPTION_WITH_ERRNO  "Fatal Error: Uncaught Exception\nThread: %s\nType: %s\nMessage: %s\nAppErrNo: %d"
diff --git a/resource.h b/resource.h
index 79b876d..70d90ca 100644
--- a/resource.h
+++ b/resource.h
@@ -39,6 +39,7 @@
 #define IDS_ELEVATED  139
 #define IDS_INSTALLEDB_VERSION            140
 #define IDS_DOWNLOAD_INCOMPLETE_EXIT      141
+#define IDS_QUERY_CORRUPT                 142
 
 // Dialogs
 
--
2.15.1

Reply | Threaded
Open this post in threaded view
|

Re: [PATCH setup] Query the user if a corrupt local file is found

Ken Brown-6
On 12/16/2017 1:37 PM, Ken Brown wrote:
> If a corrupt file is found in one of the selected mirror site
> directories, offer to delete it instead of making this a fatal error.
> Do this only on the first call to check_for cached().  If the corrupt
> file is still there on the second call, then the deletion failed, and
> the user will have to fix the problem.
>
> See https://cygwin.com/ml/cygwin/2017-12/msg00122.html for discussion.

This is unnecessarily complicated.  Please disregard.  I'll submit v2 later.

Ken

Reply | Threaded
Open this post in threaded view
|

Re: [PATCH setup] Query the user if a corrupt local file is found

Ken Brown-6
On 12/18/2017 1:40 PM, Ken Brown wrote:

> On 12/16/2017 1:37 PM, Ken Brown wrote:
>> If a corrupt file is found in one of the selected mirror site
>> directories, offer to delete it instead of making this a fatal error.
>> Do this only on the first call to check_for cached().  If the corrupt
>> file is still there on the second call, then the deletion failed, and
>> the user will have to fix the problem.
>>
>> See https://cygwin.com/ml/cygwin/2017-12/msg00122.html for discussion.
>
> This is unnecessarily complicated.  Please disregard.  I'll submit v2
> later.

Never mind.  My idea for simplification didn't work out.  I'll stick
with this version.

Ken

Reply | Threaded
Open this post in threaded view
|

Re: [PATCH setup] Query the user if a corrupt local file is found

Jon TURNEY
In reply to this post by Ken Brown-6
On 16/12/2017 18:37, Ken Brown wrote:
> If a corrupt file is found in one of the selected mirror site
> directories, offer to delete it instead of making this a fatal error.
> Do this only on the first call to check_for cached().  If the corrupt
> file is still there on the second call, then the deletion failed, and
> the user will have to fix the problem.
>
> See https://cygwin.com/ml/cygwin/2017-12/msg00122.html for discussion.

This is a nice idea.  But I think there are some structural problems
with this, though. e.g. validateCachedPackage() only checks the package
size, not hash (which happens in the install phase)

I'm also concerned about masking problems with how we got into this
state in the first place: I think either (i) a corrupt download was
stored into the cache, (ii) the valid size was changed between runs, or
(iii) the files contents actually got corrupted somehow.

(i) indicates another problem in setup

Uploading replacement packages, which would cause (ii) was permitted
historically (but of course, didn't work well), but now should be
forbidden by calm.  This could, of course, still happen with a private
package repo, and should be handled sanely.

(iii) seems unlikely, barring deliberate action.

I guess the ideal solution looks something like:

Download:
- verify size/hash of cached packages, offer to remove corrupt ones
- download packages, verifying size/hash

Install:
- verify size/hash of cached packages, skip corrupt ones
- install packages

with some memory so that we don't verify size/hash for the same package
file more than once...

> ---
>   download.cc | 17 ++++++++++++-----
>   download.h  |  4 +++-
>   res.rc      |  3 ++-
>   resource.h  |  1 +
>   4 files changed, 18 insertions(+), 7 deletions(-)
>
> diff --git a/download.cc b/download.cc
> index b606eb1..697f94a 100644
> --- a/download.cc
> +++ b/download.cc
> @@ -71,7 +71,7 @@ validateCachedPackage (const std::string& fullname, packagesource & pkgsource)
>   /* 0 on failure
>    */
>   int
> -check_for_cached (packagesource & pkgsource, bool mirror_mode)
> +check_for_cached (packagesource & pkgsource, bool mirror_mode, bool query_corrupt, HWND owner)
>   {
>     // Already found one.
>     if (pkgsource.Cached())
> @@ -114,6 +114,12 @@ check_for_cached (packagesource & pkgsource, bool mirror_mode)
>       {
>         if (validateCachedPackage (fullname, pkgsource)) >           pkgsource.set_cached (fullname);
> +      else if (query_corrupt && owner && !unattended_mode

I don't think you need to check for non-null owner here, as yesno() will
still work in that case

I kind of incline to doing the 'yes' action in unattended_mode, rather
than reporting a fatal error.
Reply | Threaded
Open this post in threaded view
|

Re: [PATCH setup] Query the user if a corrupt local file is found

Ken Brown-6
On 1/5/2018 10:00 AM, Jon Turney wrote:

> On 16/12/2017 18:37, Ken Brown wrote:
>> If a corrupt file is found in one of the selected mirror site
>> directories, offer to delete it instead of making this a fatal error.
>> Do this only on the first call to check_for cached().  If the corrupt
>> file is still there on the second call, then the deletion failed, and
>> the user will have to fix the problem.
>>
>> See https://cygwin.com/ml/cygwin/2017-12/msg00122.html for discussion.
>
> This is a nice idea.  But I think there are some structural problems
> with this, though. e.g. validateCachedPackage() only checks the package
> size, not hash (which happens in the install phase)
>
> I'm also concerned about masking problems with how we got into this
> state in the first place: I think either (i) a corrupt download was
> stored into the cache, (ii) the valid size was changed between runs, or
> (iii) the files contents actually got corrupted somehow.
>
> (i) indicates another problem in setup
>
> Uploading replacement packages, which would cause (ii) was permitted
> historically (but of course, didn't work well), but now should be
> forbidden by calm.  This could, of course, still happen with a private
> package repo, and should be handled sanely.
>
> (iii) seems unlikely, barring deliberate action.
>
> I guess the ideal solution looks something like:
>
> Download:
> - verify size/hash of cached packages, offer to remove corrupt ones
> - download packages, verifying size/hash
>
> Install:
> - verify size/hash of cached packages, skip corrupt ones
> - install packages
>
> with some memory so that we don't verify size/hash for the same package
> file more than once...

I agree.  It's strange that the code for size checking is currently in
download.cc, and the code for hash checking is currently in install.cc.
All of this checking should probably be done in a
packagesource::validate() function, which can be called during both the
download and install stages.

I'll work on this and send a new patch.

Ken