Re: Steve Marsden's WBM screenshot: The CEOP Home page for April 30, 2007 also refers to Missing Madeleine.

by HKP 02.07.15 15:56

Folks, a slight amendment to my post at 9:40 I noticed that 22/05/07 should have said 26 captures instead of the stated 37, apologies however I found my error whilst verifying my figures.

by skyrocket 02.07.15 16:18

skyrocket previously Skyrocket1 on this thread (member now) - just to keep the continuity.

One further point that has just occured to me and it's one of those you need to get down fast before you lose it!

The IA responded with an explanation that the true date for the 30/4 ceop.gov.uk home page capture was Oct 7 (if I remember correctly although the exact date in Oct is irrelevant here) and that the correct date for the mccann.htm appeal page was the 31 July. By there own admission then we are talking about TWO independent trawls (possibly by separate WBMs, as there are many - 40ish I think) that mis-dated captured pages. Not only that, but on both occasions the date stamp generated was 30/4! So now we're talking a zillion + a zillion other sites which should be affected and I would have thought about 1/zillion squared that this is what happened. The IA are talking, IMO, RUBBISH.

nah

by NickE 02.07.15 16:19

@HKP

Can you explain this link please.
[You must be registered and logged in to see this link.]

by skyrocket 02.07.15 16:22

I can spell really - please read as 'by their own' instead of 'there'!

by HKP 02.07.15 16:35

@NickE. It's court cases using Wayback as evidence, other than that I'm not sure of your question

by siobhan3443 02.07.15 16:54

10/05/07 36 urls are captured
12/05/07 50 urls are captured

@hkp

neither the index nor the calendar show a ceop capture for 7 or 10 may 2007

can you clarify what these lists are. also are you hand typing or copying and pasting?
https : //web . archive . org/web/*/http : //www . ceop . gov . uk/*
http : //web . archive . org/web/20070415000000*/ http : //ceop . gov . uk

perhaps i am mistaken, can you clarify what these numbers are of and direct me to where you found them?
thanks

by siobhan3443 02.07.15 16:56

correction -
neither the index nor the calendar show a ceop capture for 7 or 10 may 2007

sorry the above dates should read 10 or 12

by HKP 02.07.15 17:26

@siobahn. I've explained before that I used the Dr Roberts link and downloaded to excel. This was done on 29/06 and unlike some on this thread I do not resort to underhand tactics, I am quite happy to send the whole spreadsheet to admin if need be and they can verify.

by whodunit 02.07.15 17:38

Lizzy Hideho Taylor Nick Fox When a page has a link to click a downloadable poster that was not available for a while later then that page could not have existed on April 30th and is therefore a mistake..

~~~

I am not a member of the HiDeho FB group so I cannot respond over there but I know she is a member here so maybe she will see this and respond here, as she has insisted on the above several times and nobody is correcting this seriously mistaken assumption.

I am begging you..please read this FAQ section which has been quoted several times. It explains, in no uncertain terms, why a poster made at a later date was linked to on a page that existed in April.

How did I end up on the live version of a site? or I clicked on X date, but now I am on Y date, how is that possible?
Not every date for every site archived is 100% complete. When you are surfing an incomplete archived site the Wayback Machine will grab the closest available date to the one you are in for the links that are missing. In the event that we do not have the link archived at all, the Wayback Machine will look for the link on the live web and grab it if available. Pay attention to the date code embedded in the archived url. This is the list of numbers in the middle; it translates as yyyymmddhhmmss. For example in this url [You must be registered and logged in to see this link.] the date the site was crawled was Feb 29, 2000 at 12:33 and 40 seconds.

by XXXXXXXX 02.07.15 18:24

"PeterMac Today at 10:12 am

Nothing to do with the present thread, obviously !

Tony Bennett Today at 2:40 pm

Back on topic, then..."

---------------------

As it is acceptable to post off topic as long as you add a suffix to your post, I would like to ask Tony a couple of questions:

1. How did it go at Court today? I am looking forward to your report - it is a fascinating case.

2. Will your next FOI request be in Dutch?

Thanks - now back on topic.

by siobhan3443 02.07.15 18:55

why would not not use the original files which none of these dates appear in? huh, seems strange, can we all get a look at this excel file?

but then the good dr did declare chirpy insects fake link as definite proof
forgive me if i take the ramblings of an it expert who can't upload a pdf file with a pinch of salt

by SixMillionQuid 02.07.15 19:23

whodunit wrote:Lizzy Hideho Taylor Nick Fox When a page has a link to click a downloadable poster that was not available for a while later then that page could not have existed on April 30th and is therefore a mistake..

~~~

I am not a member of the HiDeho FB group so I cannot respond over there but I know she is a member here so maybe she will see this and respond here, as she has insisted on the above several times and nobody is correcting this seriously mistaken assumption.

I am begging you..please read this FAQ section which has been quoted several times. It explains, in no uncertain terms, why a poster made at a later date was linked to on a page that existed in April.

How did I end up on the live version of a site? or I clicked on X date, but now I am on Y date, how is that possible?
Not every date for every site archived is 100% complete. When you are surfing an incomplete archived site the Wayback Machine will grab the closest available date to the one you are in for the links that are missing. In the event that we do not have the link archived at all, the Wayback Machine will look for the link on the live web and grab it if available. Pay attention to the date code embedded in the archived url. This is the list of numbers in the middle; it translates as yyyymmddhhmmss. For example in this url [You must be registered and logged in to see this link.] the date the site was crawled was Feb 29, 2000 at 12:33 and 40 seconds.

The bit I've underlined indicates that a a CEOP page was crawled / archived on 30 April 2007.

by HKP 02.07.15 19:31

@siobahn. Firstly, the reason I downloaded it to excel is for ease of analysis, the data can then be easily filtered and sorted by anything that is in the cells, searchability is also greatly increased. Can you get to see the file, well its over a megabyte so not easy to post, I wouldn't send it to any unstrusted source so that rules you out sorry.

You seem to be taking cheap shots and trying to discredit me (I never opened your links as it is probably Syn having another laugh she said yesterday that she'd done another). Finally show me where I claimed, said or even intimated that I'm an IT expert I'm not and that's all that you need to know. No doubt your sidekick will be along soon with her subterfuge.

I stand by my analysis its been done as hard endeavour looking for some sort of answer, read my post again and enjoy!

by whodunit 02.07.15 19:38

The April 30 date is not only in the date code of the url, it was embedded in the page itself, it is NOW embedded in 'next/previous capture' codes of the next/previous captured pages, and it is listed in the index.

I honestly do not know what more people need to accept that mccann.html was captured on April 30, 2007. All arguments 'against' are merely massive walls of confusion, obfuscation, and distraction.

As Agnos said at Only in America:

"If a person purports to know the class of "error" that has generated the date code in question; and yet that person is unable to assert the extent and duration of said "error" across the WBM structure, then that person does not know the class of "error" at all."

It is not for proponents of the 'early' existence of the page to prove what is self evident. It is for the debunkers to prove that it did not exist.

by HKP 02.07.15 19:41

@SixMillionQuid. Not meant disrespectfully but that FAQ from Wayback has been posted quit a few times over the piece it's just that some don't want to pay any attention.

by **Tony Bennett** 02.07.15 20:19

XXXXXXXX wrote:1. How did it go at Court today? I am looking forward to your report - it is a fascinating case.

REPLY: Les Balkwell asked the judge for permission to amend (by which I mean 'add to', not change) his original pleadings in 23 different respects.

He was allowed 16 additions, the other 7 were refused for legal reasons

2. Will your next FOI request be in Dutch?

REPLY: No.

by **Tony Bennett** 02.07.15 20:39

Syn wrote:
Tony Bennett wrote:
Syn wrote:Problem with subsets says Chris Butler....

.... that brings us right back to the beginning where I posted this on 17th June when this merry go round first started.

I have now underlined the relevant part

[You must be registered and logged in to see this image.]

I shall try and help :)

1. how many hundreds/thousands/tens of thousands/hundreds of thousands of Wayback records are false because of this faulty subset

I have no idea, what makes you think that I would know? What I can be logically sure of is that it will not be specific to just 20070430 archives. Millions of pages crawled and archived on 20070430 and many other dates will have been archived incorrectly I'm sure. However, the millions affected will be a tiny, tiny percentage give how many website WB crawls each year. There is no way to ascertain how many of the millions of websites out there in 2007 and probably prior and beyond 2007 which were crawled and archived by WB had pdf links, flash player, needed server side verification for SSIs etc are out there that may be incorrect due to the claimed subset issue. Wayback are not going to tell us which other sites were affected and identifying similarly affected websites would be like trying to find a needle in a haystack for us mere mortals. We are only able to identify an anomaly with the CEOP pages because we have an identifier of 20070503 as the date that it is claimed that Madeleine went missing. Therefore with this point of reference it is possible to cross reference any archived content. If you have any idea how to identify any other website that may have been similarly affected regardless of date, please do let me know as I cannot fathom any. There is no point in saying ohh Amazon or the BBC was archived on 20070430 with no problem at all as they may not have had any pdf links, flash content or server side SSIs or anything else that may have made archiving a website go squewiffy once WB appended their server side javascript to a subset of the pages html. Not looked but chances are that Amazon. BBC webpages etc were not using simple HTML but were using php, ruby on rails, asp etc back then to dynamically generate relevant html content and will have been unaffected.

2. when did Wayback discover this 'subset error'

Again, I have no idea, I simply found info re Wayback Machine/archive.org that explains the issue that they have had/may still be having regarding inserting/appended javascript into html pages which contain pdfs, flash etc and their acknowledgement that such a process is not the most efficient and issues regarding timestamps and rendering of data can occur and why they deprecate and would ideally wish to discontinue utilising said processes. Many webpages will have been archived correctly and as much as a great many webpages may have been affected by a similar timestamp subset issue, many millions more will be perfectly fine.

3. over what period did this subset error persist?

I have no idea. It would seem that it has been an ongoing issue since implementation.

@ Syn I am really grateful for the answers, thank you.

But in my attempts to understand the truth in all this, the comments I've bolded and underlined in red in your post cause me renewed difficulty in accepting that you have the right answer.

Especially so with your third and last answer, where you say that 'it' [by which I think you mean the 'subset error'] 'has been an ongoing issue since implementation'. That sounds like a very long time indeed.

You see, those that assert that this is 'an error' do not (unless I am mistaken) know what kind of error it was yet. Even Wayback's Christopher Butler is reported (via Lizzy Taylor) to merely say it was some sort of subset error.

Unless someone can tell me WHAT kind of error this was, HOW OFTEN it has occurred, HOW LONG it has lasted for, WHETHER it is a susbet error or a coding error or what, and WHETHER it has been corrected, it could be for all I know that Wayback Machine has millions or zillions of errors and is therefore worthless.

And again, if someone says: "It's an error, can't you see, it's obvious", or "It's an error, just accept it", or "It's some sort of system error, just forget about it now and go away", that isn't nearly enough to persuade me that 'it's just a glitch'

by XXXXXXXX 02.07.15 20:42

Thanks for your replies Tony.

by rustyjames 02.07.15 20:51

Trying to catch up with this thread having been away at Glastonbury completely out of touch for 6 days.

I've previously posted how to search the indexes using the Internet Archive's API into their CDX.

I've done a search on all [You must be registered and logged in to see this link.] pages indexed for the whole of 2007. If you want to do the same the link is below:
[You must be registered and logged in to see this link.]

I've loaded the results into excel and used pivot tables to analyse it and get some stats.

There are 13130 entries in the index for 2007.

Pages were indexed from ceop on 253 separate days during 2007. That is an average of 51.89 per day.

However there are 3786 entries for 30th April. Excluding those that is an average of 37.07 per day for the remaining days.

Now the interesting thing is breaking it down to the second in the index.

In that case there are 8732 unique timestamps over the 253 days.

Excluding the 30th April, the maximum number of indexed pages per second is 3, and an average of 1.07.

Yet on the 30th April every single one of the 3786 entries are timestamped in the same second - i.e. 20070430115803

That makes no sense whatsoever, so I'm now fairly certain there is at least an indexing problem, and as I've said a number of times before, what we really need are the original ".arc" files. I've looked for alternative sources but there is nothing in the public domain.

There is a possible alternative source for indexes here - [You must be registered and logged in to see this link.] but the 2007 data is a 26Gb download of tens of thousands of unsorted files.

by **Tony Bennett** 02.07.15 20:54

rustyjames wrote:Yet on the 30th April every single one of the 3786 entries are timestamped in the same second - i.e. 20070430115803

That makes no sense whatsoever, so I'm now fairly certain there is at least an indexing problem, and as I've said a number of times before, what we really need are the original ".arc" files. I've looked for alternative sources but there is nothing in the public domain.

There is a possible alternative source for indexes here - [You must be registered and logged in to see this link.] but the 2007 data is a 26Gb download of tens of thousands of unsorted files.

Could that be classed as either...

'a subset error'

or

'a coding error'?

TIA

by HKP 02.07.15 21:05

@Rustyjames. Ha, ha I've just spent ages looking at a download that had almost 9000 URLs over the life of ceop!!! The link came from the Dr Martin only in America blog site which I think everyone else was using too. With that I too was using pivots etc. but will now need to look at what you've got here (not doubting in any way just for my own benefit) I'll keep a watch out for your further analysis thanks.

by rustyjames 02.07.15 21:09

HKP wrote:@Rustyjames. Ha, ha I've just spent ages looking at a download that had almost 9000 URLs over the life of ceop!!! The link came from the Dr Martin only in America blog site which I think everyone else was using too. With that I too was using pivots etc. but will now need to look at what you've got here (not doubting in any way just for my own benefit) I'll keep a watch out for your further analysis thanks.

Can you give me a link to the Dr Martin link. Searched and can't spot it.

The link in my previous post is easy enough to save as a text file and import into excel as a space delimited file.

Edited to add: Could someone also point me to the original reference to a "subset error"?

by HKP 02.07.15 21:11

@siobahn
Just to prove a point I snapshot the 10 &12 May captures at this link.
http ://i18. servimg . com/u/f18/16/26/76/90/screen10 .png
(take out spaces, guests can't link)

by HKP 02.07.15 21:21

@Rustyjames.
Can't do links, go to comments section fourth one down

by rustyjames 02.07.15 21:27

HKP wrote:@Rustyjames.
Can't do links, go to comments section fourth one down

OK got it - not as useful as it just has the date rather than a full timestamp.

Steve Marsden's WBM screenshot: The CEOP Home page for April 30, 2007 also refers to Missing Madeleine.