Re: Claim by 'Stevo' - "CEOP show Maddie is missing on 30th April 2007"

by Joss 20.06.15 13:26

Syn wrote:
whatsupdoc wrote:Syn Could you explain the following link which arrives at 30th April having a 1st May @ midnight in the date / time stamp followed by the * ?

[You must be registered and logged in to see this link.]
Looking at it whilst they are retrofitting code or whatever they are doing is a waste of time at the moment to be honest. They have said once they re-index the 30th April date will go. Not sure how often they do that though.

BBM, Where was that said, and by whom & Why?

by Guest 20.06.15 13:29

Wayback Machine was very accurate in April 2007!

[You must be registered and logged in to see this image.]

by Nuala 20.06.15 13:33

@ BlueBag

You're working hard on this aren't you.

How rude.

As I said originally, Tony Bennett asked a question and because I knew the answer I signed up to answer him.

I also took the time and trouble to try and explain to you how web crawlers work as well.

If Tony, or anyone else who is civil, has any further questions, I will try and help.

Any more rudeness and I'm gone, I don't have to put up with that when my only motivation was to try and help answer a question because it's a subject I happen to know a lot about.

by **Jill Havern** 20.06.15 13:42

This thread has now been moved into the new section where guests can contribute technical knowledge to the debate without the need to register.

I've tweeted the link to Jim Gamble... titter

by **Tony Bennett** 20.06.15 13:52

Nuala wrote:@ BlueBag

You're working hard on this aren't you.

How rude.

As I said originally, Tony Bennett asked a question and because I knew the answer I signed up to answer him.

I also took the time and trouble to try and explain to you how web crawlers work as well.

If Tony, or anyone else who is civil, has any further questions, I will try and help.

Any more rudeness and I'm gone, I don't have to put up with that when my only motivation was to try and help answer a question because it's a subject I happen to know a lot about.

@ Nuala I fully agree - BlueBag had overstepped the mark with his response to you.

Speaking personally, I am grateful for all responses that help us to shed further light on this 'CEOPGate' mystery, and I thank you again for intervening.

I think however that claim and counter-claim by 'tecchies' must be tested.

In a nutshell, you said that web-crawlers could not 'find' dummy web pages that folk were workng on, but had not yet 'gone live' yet - which you said meant that there was as yet no 'hyperlink'.

You have probably seen the quote I gave from 'Resistor', timed at 11.37, from another forum, where he says that some search engines can pick up pages which do not have hyperlinks.

Is he plain wrong about that?

by Syn 20.06.15 13:54

Joss wrote:
Syn wrote:
whatsupdoc wrote:Syn Could you explain the following link which arrives at 30th April having a 1st May @ midnight in the date / time stamp followed by the * ?

[You must be registered and logged in to see this link.]
Looking at it whilst they are retrofitting code or whatever they are doing is a waste of time at the moment to be honest. They have said once they re-index the 30th April date will go. Not sure how often they do that though.
BBM, Where was that said, and by whom & Why?

Flying visit as really do have to zoom out now but will be back later :)

Chris followed up his email to Isabelle McFadden with this:

[You must be registered and logged in to see this link.]

by Joss 20.06.15 14:03

Syn wrote:
Joss wrote:
Syn wrote:
whatsupdoc wrote:Syn Could you explain the following link which arrives at 30th April having a 1st May @ midnight in the date / time stamp followed by the * ?

[You must be registered and logged in to see this link.]
Looking at it whilst they are retrofitting code or whatever they are doing is a waste of time at the moment to be honest. They have said once they re-index the 30th April date will go. Not sure how often they do that though.
BBM, Where was that said, and by whom & Why?
Flying visit as really do have to zoom out now but will be back later :)

Chris followed up his email to Isabelle McFadden with this:

[You must be registered and logged in to see this link.]

Thanks.

by sar 20.06.15 14:05

weird when a "100 pager" like this happens, lurkers come out in force to dispute, nada before the spike....

by worriedmum 20.06.15 14:06

So are the next questions to Wayback,
WHY?
HOW?
WHEN?

by Nuala 20.06.15 14:09

@ Tony Bennett

Is he plain wrong about that?

Yes he is.

I think however that claim and counter-claim by 'tecchies' must be tested.

I agree, and understand how frustrating it is being caught between two opposing "experts" and not knowing what to believe.

There is a very simple way of sorting this issue, and that's to ask Wayback Machine exactly what their web crawlers were able to access back in 2007.

An answer from them, as I see it, would be the only way to get some sort of reassurance on the issue, particularly for the non-techies.

I stand by what I've said though re how web crawlers work. Web crawlers aren't hackers, they're bots designed to do a particular job. In the case of Wayback Machine the job of their crawlers was to take a snapshot of how a web page looked at a certain moment in time. Whatever else might have been on a web server at that moment, and unconnected via a hyperlink to the website in question, the crawler would have known nothing about, and certainly wouldn't be trying to hack the server to find out.

by **Tony Bennett** 20.06.15 14:09

Syn wrote:
[You must be registered and logged in to see this link.]

17 JUNE

1. Our systems made a mistake

2. Er, because it's a mistake, it really should not be in there in our archive

3. So what we've done now is to remove this system error from our archive

4. We need to fix this with Wayback Machine

5. And then we will amend our archive so that it doesn't say what it did on Wednesday morning

6. We are currently investigating further and will get back to you

7. The 'blue circle' thingy is also wrong

8. That'll have to wait a bit longer i.e until the next indexing of the Wayback Machine

9. As I say, once we've sorted out all the technical issues about this regrettable error, we'll get back to you

20 JUNE, 2.10pm

10. Still investigating this, awfully sorry for the delay. This error is proving quite a hard task for us to explain at the moment. Rest assured, as soon as we're in a position to make a statement about it, we will! Be patient

by sallypelt 20.06.15 14:13

And to think that CB was so confident in the accuracy of WBM in his first email to Isabella, yet a few hours later, in a hurried email to Liz, it was an "Error".

Ummm!

by sallypelt 20.06.15 14:15

Tony Bennett wrote:
Syn wrote:
[You must be registered and logged in to see this link.]
17 JUNE

1. Our systems made a mistake

2. Er, because it's a mistake, it really should not be in there in our archive

3. So what we've done now is to remove this system error from our archive

4. We need to fix this with Wayback Machine

5. And then we will amend our archive so that it doesn't say what it did on Wednesday morning

6. We are currently investigating further and will get back to you

7. The 'blue circle' thingy is also wrong

8. That'll have to wait a bit longer i.e until the next indexing of the Wayback Machine

9. As I say, once we've sorted out all the technical issues about this regrettable error, we'll get back to you

20 JUNE, 2.10pm

10. Still investigating this, awfully sorry for the delay. This error is proving quite a hard task for us to explain at the moment. Rest assured, as soon as we're in a position to make a statement about it, we will! Be patient

This error is proving quite a hard task for us to explain at the moment

so, "at the moment" the "error" is unexplainable.

by sallypelt 20.06.15 14:21

[You must be registered and logged in to see this link.]

by **Tony Bennett** 20.06.15 14:21

sallypelt wrote:And to think that CB was so confident in the accuracy of WBM in his first email to Isabella, yet a few hours later, in a hurried email to Liz, it was an "Error".

Ummm!

Yep, I think the French have it with 'Volte Face'. Or as the English would say, 'A 180-degree turn'.

@ Nuala

I think I fully understand your point BUT I'm not sure you've fully answered the point made earlier today by MRNOODLES (page 31 of this thread, for reference), where he discusses the 'Shirt-gate' incident at Norwich City Football Club.

If an ordinary bloke with no tecchie experience can go into a website and - clearly - has found a page there that was not 'live' and had not been uploaded yet to the website - why cannot a web crawler/search engine, especially a very well-designed one, find (in a similar way) a page, like the CEOP 'Madeleine' page, that has not yet been uploaded to the main site?

by Guest 20.06.15 14:23

Tony Bennett wrote:
Nuala wrote:@ BlueBag

You're working hard on this aren't you.

How rude.

As I said originally, Tony Bennett asked a question and because I knew the answer I signed up to answer him.

I also took the time and trouble to try and explain to you how web crawlers work as well.

If Tony, or anyone else who is civil, has any further questions, I will try and help.

Any more rudeness and I'm gone, I don't have to put up with that when my only motivation was to try and help answer a question because it's a subject I happen to know a lot about.
@ Nuala I fully agree - BlueBag had overstepped the mark with his response to you.

Speaking personally, I am grateful for all responses that help us to shed further light on this 'CEOPGate' mystery, and I thank you again for intervening.

I think however that claim and counter-claim by 'tecchies' must be tested.

In a nutshell, you said that web-crawlers could not 'find' dummy web pages that folk were workng on, but had not yet 'gone live' yet - which you said meant that there was as yet no 'hyperlink'.

You have probably seen the quote I gave from 'Resistor', timed at 11.37, from another forum, where he says that some search engines can pick up pages which do not have hyperlinks.

Is he plain wrong about that?

I am suspicious of "new" members signing up to put us right on a hot topic.. we've had two in the last 24 hours.

by Guest 20.06.15 14:28

Tony Bennett wrote:
sallypelt wrote:And to think that CB was so confident in the accuracy of WBM in his first email to Isabella, yet a few hours later, in a hurried email to Liz, it was an "Error".

Ummm!
Yep, I think the French have it with 'Volte Face'. Or as the English would say, 'A 180-degree turn'.

@ Nuala

I think I fully understand your point BUT I'm not sure you've fully answered the point made earlier today by MRNOODLES (page 31 of this thread, for reference), where he discusses the 'Shirt-gate' incident at Norwich City Football Club.

If an ordinary bloke with no tecchie experience can go into a website and - clearly - has found a page there that was not 'live' and had not been uploaded yet to the website - why cannot a web crawler/search engine, especially a very well-designed one, find (in a similar way) a page, like the CEOP 'Madeleine' page, that has not yet been uploaded to the main site?

But Nuala has already said it's impossible so it must be impossible because Nuala knows exactly how the Way BAck machine program worked in 2007 and that's why he/she signed on urgently to tell us.

Still no real explantation from the real people.

by sallypelt 20.06.15 14:28

Can any technician explain, in simple terms, what Web scraping is?

by **Tony Bennett** 20.06.15 14:29

BlueBag wrote:
I am suspicious of "new" members signing up to put us right on a hot topic...we've had two in the last 24 hours.

@ BlueBag

At least it's nowhere near as bad as that evening in late October 2013 after I had first posted my major doubts about the 'Smithman' sighting, and tentatively suggested that maybe the Smiths had fabricated their claimed 'sighting'.

That evening, we had SIX new members join the forum.

Each one of those six ONLY JOINED THE SMITHMAN THREAD.

And every single one of those six subsequently proved to be a disruptor

by Guest 20.06.15 14:30

sallypelt wrote:Can any technician explain, in simple terms, what Web scraping is?

It's just copying.

It's copying a webpage at a certain date and time and archiving it.

A snap shot.

Scraping is supposed to sound more down with the kids.

by sallypelt 20.06.15 14:31

Ion disperses data harvesting activity over our vast network of anonymous IP addresses, allowing you open access and drastically reducing your collection time.
[You must be registered and logged in to see this image.]

by jeanmonroe 20.06.15 14:43

Get'emGonçalo wrote:

I've tweeted the link to Jim Gamble...

How very 'hurtful and unhelpful' for Jimbo, that will be! laughat

by Gaggzy 20.06.15 14:46

Tony Bennett wrote:
Syn wrote:
Wayback has issues loading the flag gifs on all captures of the page including the erroneous 30/04/07 one....I hope this makes sense
YES.

It does.

You are saying that Wayback Machine/archive.org is unreliable.

Period.

Well, well, well.

UNRELIABLE dogs.

UNRELIABLE archive.org.

UNRELIABLE Portuguese police.

If 'those two' held a live press conference outside 10 Downing Street to tell the world about what really happened to Madeleine and their involvement ..... I could imagine Cameron jumping in to tell the viewers that this 'confession' is completely UNRELIABLE. (wahhay, here's that word again).

by sallypelt 20.06.15 14:47

"stealing information" with Webscraper:

[You must be registered and logged in to see this link.]

There are many secure websites out there that provide useful information but do not have a public API to access it's data. A prime example of this is the LinkedIn website. You might love to gather some info from LinkedIn, but their promise to deliver a public API has yet to come to fruition. The problem is, the pages with all the good data are secure, requiring the user to log in before accessing these pages. Let's say we want to scrape this data from these pages programatically? We need to authenticate to access these pages. We can do that by reusing the authentication cookie from the site that we receive when we log in with a browser.

I haven't got a clue if this information is of any value to this thread, but I am posting it, anyway, in case it's dynamite

Opps, link: [You must be registered and logged in to see this link.]

by Nuala 20.06.15 14:52

@ Tony Bennett

I think I fully understand your point BUT I'm not sure you've fully answered the point made earlier today by MRNOODLES (page 31 of this thread, for reference), where he discusses the 'Shirt-gate' incident at Norwich City Football Club.

If an ordinary bloke with no tecchie experience can go into a website and - clearly - has found a page there that was not 'live' and had not been uploaded yet to the website - why cannot a web crawler/search engine, especially a very well-designed one, find (in a similar way) a page, like the CEOP 'Madeleine' page, that has not yet been uploaded to the main site?

Note that this isn't to do with uploading to the main site, the page IS uploaded, it's a link to the page that connects it to the main site that we're discussing. You don't upload a page to a website, you upload a page to the web host and then "connect" it via a hyperlink to the main website so it becomes part of the main website.

Just wanted to clarify that because the "uploaded" word is confusing the issue.

So, to your question, if someone looks at the coding for a web page and finds a link there, one not currently visible on the website itself, then a crawler will find it as well, it's as simple as that.

Your original question though, from way back, was whether a crawler could find a page not currently linked, a hidden page, ready to be connected to the main website at some point, and the answer to that is no, a crawler can't find a page that doesn't have a hyperlink pointing to it.

In the case of ShirtGate there IS a hyperlink, which both a person and crawler can find.

by Nuala 20.06.15 14:56

But Nuala has already said it's impossible so it must be impossible because Nuala knows exactly how the Way BAck machine program worked in 2007 and that's why he/she signed on urgently to tell us.

Still no real explantation from the real people.

Right, that's it, I'm gone.

I'm not putting up with such rudeness when all I'm doing is trying to help.

by jeanmonroe 20.06.15 14:59

(WBM) Wayback Machine.

"This error is proving quite a hard task for us to explain at the moment"
------------------------------------------------------
OG. ('abduction 'solving' specialists')

"This 'abduction', of Madeleine, is proving quite a hard task for us to explain at the moment."

by Joss 20.06.15 15:06

Surely the Wayback Machine can't be that hard to understand. Think i need to search for the book "The Wayback Machine for Dummies", or go look for the Tardis big grin

Its just an internet library isn't it?

by **Richard D. Hall** 20.06.15 15:06

In order to understand what the WBM does, it is necessary to have a print out of their servers source code, which was running on 30 April 2007. This is something that WBM probably have archived somewhere and could look at. As far as I am aware nobody other than them have visibility of this code, therefore only they know exactly (and I mean EXACTLY) how their crawler functions. If you were to get the code and do a code walkthrough I suspect the code will show that their systems seeks out new webpages, and newly updated webpages. When it finds a new webpage (such as mccann.html) on the ceop website, it will invoke code to copy that page onto a folder on its server. In order to copy the new file it will have a function to create the new folder. This function I am sure will use the systems real time clock to construct the folder name based on the current time and day. Once the folder is made on their server, the file is then copied. Their system also ammends the file by adding a date and time stamp within the file and also some other information it might need at a later time. My description here is a general one because I do not have a copy of their source code. I think it is unlikely that a walkthrough of their source code would reveal that it is possible to create a folder with a date which pre-dates the date when a new file is discovered.

by jeanmonroe 20.06.15 15:07

Nuala wrote:But Nuala has already said it's impossible so it must be impossible because Nuala knows exactly how the Way BAck machine program worked in 2007 and that's why he/she signed on urgently to tell us.

Still no real explantation from the real people.

Right, that's it, I'm gone.

I'm not putting up with such rudeness when all I'm doing is trying to help.

Just chill, Nuala.

You know how many people have been 'rude' to me over the years?

NONE!

They wouldn't 'dare'! laughat

I'm just getting to grips with 'Webscrumping', mentioned earlier.

Does that mean FREE 'virtual' apples?