DistFileSurvey Results

There are several different representations for the results of the distfile survey. For technical background on how the distfile survey works, see DistfileSurvey.

Standard Results

The main URL

The results page for a given port are broken down by file and URL. When a file is fetchable from all of its possible URLs, it will say something like

A port is considered fetchable if all of its files are fetchable from at least one site. However, user experience is best when all files are fetchable from all sites. If a given file is fetchable from only some of the sites, it will look something like

The 4 bad, while not fatal, may be the first 4 that "make fetch" tries. It's best to try to make this zero.

If a given file is not fetchable from any of the sites, it will look something like

This port is effectively unfetchable, except possibly from the backup master site.

For these last 2 results, the actual results for each URL will be shown underneath the "File:" line, such as: 

http://www.ibiblio.org/pub/Linux/system/backup/afio-2.4.7.tgz: 404 Not Found (Last OK result Tue Dec 16 12:42:50 2003 ) 
http://www.gtlib.cc.gatech.edu/pub/Linux/system/backup/afio-2.4.7.tgz: 404 Not Found (Last OK result Tue Dec 16 15:45:21 2003 ) 
ftp://sunsite.cnlab-switch.ch/mirror/linux/sunsite/system/backup/afio-2.4.7.tgz: looking for file 550 afio-2.4.7.tgz: No such file or directory. (Last OK result Sun Dec 14 21:54:34 2003 ) 
ftp://ftp.cs.tu-berlin.de/pub/linux/Mirrors/sunsite.unc.edu/system/backup/afio-2.4.7.tgz: looking for file 550 afio-2.4.7.tgz: No such file or directory. (Last OK result Fri Dec 12 0:50:52 2003 ) 
ftp://ftp.physics.auth.gr/pub/mirrors/ibiblio/Linux/system/backup/afio-2.4.7.tgz: looking for file 550 afio-2.4.7.tgz: No such file or directory. (Last OK result NEVER [checked 8 times since Fri Jul 2 23:58:28 2004 , last time was Sat Oct 1 21:22:06 2005 ]) 
ftp://ftp.edisontel.com/pub/Sunsite_Mirror/system/backup/afio-2.4.7.tgz: looking for file 550 afio-2.4.7.tgz: No such file or directory. (Last OK result NEVER [checked 6 times since Sun Sep 11 11:36:07 2005 , last time was Sat Oct 1 14:36:29 2005 ]) 
ftp://ftp.nluug.nl/pub/metalab/system/backup/afio-2.4.7.tgz: looking for file 550 afio-2.4.7.tgz: No such file or directory. (Last OK result NEVER [checked 7 times since Thu Jul 15 2:02:11 2004 , last time was Sat Oct 1 20:46:26 2005 ]) 
ftp://ftp.lip6.fr/pub/linux/sunsite/system/backup/afio-2.4.7.tgz: looking for file 450 afio-2.4.7.tgz: No such file or directory (Last actual result Fri Jan 16 11:51:04 2004 looking for file 550 afio-2.4.7.tgz: No such file or directory.) (Last OK result Wed Dec 17 19:08:26 2003 ) 
ftp://ftp.nvg.ntnu.no/pub/mirrors/metalab.unc.edu/system/backup/afio-2.4.7.tgz: looking for file 550 afio-2.4.7.tgz: No such file or directory. (Last OK result Sun Dec 14 10:22:37 2003 ) 
ftp://ftp.icm.edu.pl/pub/Linux/sunsite/system/backup/afio-2.4.7.tgz: looking for file 550 afio-2.4.7.tgz: No such file or directory. (Last OK result Fri Dec 12 6:25:16 2003 ) 
ftp://ftp.cse.cuhk.edu.hk/pub4/Linux/system/backup/afio-2.4.7.tgz: looking for file 550 afio-2.4.7.tgz: No such file or directory. (Last OK result Thu Apr 29 6:09:25 2004 ) 
ftp://ftp.kddlabs.co.jp/Linux/metalab.unc.edu/system/backup/afio-2.4.7.tgz: looking for file 550 afio-2.4.7.tgz: No such file or directory. (Last OK result Sun Dec 14 5:03:55 2003 ) 
ftp://ftp.chg.ru/pub/Linux/sunsite/system/backup/afio-2.4.7.tgz: looking for file 550 afio-2.4.7.tgz: No such file or directory. (Last OK result Sat Dec 13 20:27:39 2003 )

Note that when the file had previously been successfully found at this site, it'll tell you when it was - this can help to determine if something is a temporary problem or if it's gone for good. (For most of these sites, it had been found before but the most recent time was over a year ago; for ftp.nluug.nl it never saw the file since it started checking it)

"Bad" results will be in red, "OK" results are in green, and "Skipped" results are in mauve. I'd love a more compact representation for this data.

If there are multiple sites, and some sites have zero files fetchable, a table will be shown under the port's results listing all the sites and the number of distfiles fetchable from each site.

(To be honest, "0 OK, 10 bad" is not fatal, since the file likely exists on the FreeBSD backup master site; this is all about improving user experience.

Results by maintainer

Results by maintainer

This is the same data as is used to send the biweekly emails (sent on the 7th and 21st).

The summary looks like:

fenner@freebsd.org 2 unfetchable ports(38 ok, 8 bad, 2 skipped)

The 2 unfetchable ports means that at least one file for each port is fetchable from none of its master sites, forcing a fallback to the FreeBSD distfiles mirrors. These are the first problems that should be investigated; however, as described above, the 8 bad should be investigated - perhaps they point to bad master sites, or other related problems.

2 skipped means that the distfile survey didn't know what to do with the result it got. Often this is a timeout or other connection problem, and if it persists should also be investigated.

Custom Views

Pav's Page

Pav's page

Pav asked about the WWW: URLs from pkg-descr specifically. This page lists each non-success from the last run.

public_distfiles report

public_distfiles report

This lists out files that are supposed to be in MASTER_SITE_LOCAL but aren't.

Some possible reasons for being on this page:

Results by site

Results by site

This page is good for finding, e.g., sites used by a lot of ports that have reorganized, or for fixing entries in bsd.sites.mk

Grouping helper

Grouping helper

This shows a matrix of file x site; green = good, red = bad, white = that file isn't fetched from that site. If you see a line of green and a swath of red, it may be a port that could use the group syntax to reduce the number of bad URLs (the red ones). Of course, distfile survey false negatives and false positives can make a smattering of confusing entries in the tables, so it's best to verify that what the table shows is really right before going for any changes.

Its tables look like:

file1

file2

file3

file4

file5

site1

+

-

-

-

-

site2

+

-

-

-

-

site3

+

-

-

-

-

site4

-

+

+

+

+

This is an example of a perfectly obvious requirement - file1 is fetchable only from site1, site2 and site3, and file2, file3, file4 and file5 are only fetchable from site4. This means that you should create two groups: put site1, site2 and site3 in one group, and site4 in the other; put file1 in the first group, and file2-5 in the second. 

MASTER_SITES= site1:g1 site2:g1 site3:g1 site4:g2
DISTFILES= file1:g1 file2:g2 file3:g2 file4:g2 file5:g2

If site1-3 are in a MASTER_SITE_FOO, this is done by: 

MASTER_SITES= ${MASTER_SITE_FOO:S/$/:g1/} site4:g2

If you have two different MASTER_SITE_FOOs, and they need different subdirectories, this can be done too: 

MASTER_SITES= ${MASTER_SITE_LOCAL:S/$/:local/} ${MASTER_SITE_SOURCEFORGE:S/$/:sf/}
MASTER_SITE_SUBDIR= fenner/:local apple2hylafax/:sf

NOTE that the MASTER_SITE_SUBDIR requires a trailing slash in this case, and only this case.

If the same file is being fetched from multiple MASTER_SITE groups, but you still need to use grouping to get the subdirectories right, as in the above example, you can use 

DISTFILES= ${DISTNAME}${EXTRACT_SUFX}:local,sf

Results sorted by bad files

http://people.freebsd.org/~fenner/portsurvey/crazy.html

Lists in order of #bad URLs; these files are fetchable but some have, e.g., 1 good and 100 bad URLs so also need to be looked at.

CategoryHistorical

DistFileSurveyResults (last edited 2015-10-24T07:26:22+0000 by MarkLinimon)