Re: Claim by 'Stevo' - "CEOP show Maddie is missing on 30th April 2007"

by sallypelt 20.06.15 14:31

Ion disperses data harvesting activity over our vast network of anonymous IP addresses, allowing you open access and drastically reducing your collection time.
[You must be registered and logged in to see this image.]

by jeanmonroe 20.06.15 14:43

Get'emGonçalo wrote:

I've tweeted the link to Jim Gamble...

How very 'hurtful and unhelpful' for Jimbo, that will be! laughat

by Gaggzy 20.06.15 14:46

Tony Bennett wrote:
Syn wrote:
Wayback has issues loading the flag gifs on all captures of the page including the erroneous 30/04/07 one....I hope this makes sense
YES.

It does.

You are saying that Wayback Machine/archive.org is unreliable.

Period.

Well, well, well.

UNRELIABLE dogs.

UNRELIABLE archive.org.

UNRELIABLE Portuguese police.

If 'those two' held a live press conference outside 10 Downing Street to tell the world about what really happened to Madeleine and their involvement ..... I could imagine Cameron jumping in to tell the viewers that this 'confession' is completely UNRELIABLE. (wahhay, here's that word again).

by sallypelt 20.06.15 14:47

"stealing information" with Webscraper:

Scraping, or Programatically Accessing, a Secure Webpage

There are many secure websites out there that provide useful information but do not have a public API to access it's data. A prime example of this is the LinkedIn website. You might love to gather some info from LinkedIn, but their promise to deliver a public API has yet to come to fruition. The problem is, the pages with all the good data are secure, requiring the user to log in before accessing these pages. Let's say we want to scrape this data from these pages programatically? We need to authenticate to access these pages. We can do that by reusing the authentication cookie from the site that we receive when we log in with a browser.

I haven't got a clue if this information is of any value to this thread, but I am posting it, anyway, in case it's dynamite

Opps, link: [You must be registered and logged in to see this link.]

by Nuala 20.06.15 14:52

@ Tony Bennett

I think I fully understand your point BUT I'm not sure you've fully answered the point made earlier today by MRNOODLES (page 31 of this thread, for reference), where he discusses the 'Shirt-gate' incident at Norwich City Football Club.

If an ordinary bloke with no tecchie experience can go into a website and - clearly - has found a page there that was not 'live' and had not been uploaded yet to the website - why cannot a web crawler/search engine, especially a very well-designed one, find (in a similar way) a page, like the CEOP 'Madeleine' page, that has not yet been uploaded to the main site?

Note that this isn't to do with uploading to the main site, the page IS uploaded, it's a link to the page that connects it to the main site that we're discussing. You don't upload a page to a website, you upload a page to the web host and then "connect" it via a hyperlink to the main website so it becomes part of the main website.

Just wanted to clarify that because the "uploaded" word is confusing the issue.

So, to your question, if someone looks at the coding for a web page and finds a link there, one not currently visible on the website itself, then a crawler will find it as well, it's as simple as that.

Your original question though, from way back, was whether a crawler could find a page not currently linked, a hidden page, ready to be connected to the main website at some point, and the answer to that is no, a crawler can't find a page that doesn't have a hyperlink pointing to it.

In the case of ShirtGate there IS a hyperlink, which both a person and crawler can find.

by Nuala 20.06.15 14:56

But Nuala has already said it's impossible so it must be impossible because Nuala knows exactly how the Way BAck machine program worked in 2007 and that's why he/she signed on urgently to tell us.

Still no real explantation from the real people.

Right, that's it, I'm gone.

I'm not putting up with such rudeness when all I'm doing is trying to help.

by jeanmonroe 20.06.15 14:59

(WBM) Wayback Machine.

"This error is proving quite a hard task for us to explain at the moment"
------------------------------------------------------
OG. ('abduction 'solving' specialists')

"This 'abduction', of Madeleine, is proving quite a hard task for us to explain at the moment."

by Joss 20.06.15 15:06

Surely the Wayback Machine can't be that hard to understand. Think i need to search for the book "The Wayback Machine for Dummies", or go look for the Tardis big grin

Its just an internet library isn't it?

by **Richard D. Hall** 20.06.15 15:06

In order to understand what the WBM does, it is necessary to have a print out of their servers source code, which was running on 30 April 2007. This is something that WBM probably have archived somewhere and could look at. As far as I am aware nobody other than them have visibility of this code, therefore only they know exactly (and I mean EXACTLY) how their crawler functions. If you were to get the code and do a code walkthrough I suspect the code will show that their systems seeks out new webpages, and newly updated webpages. When it finds a new webpage (such as mccann.html) on the ceop website, it will invoke code to copy that page onto a folder on its server. In order to copy the new file it will have a function to create the new folder. This function I am sure will use the systems real time clock to construct the folder name based on the current time and day. Once the folder is made on their server, the file is then copied. Their system also ammends the file by adding a date and time stamp within the file and also some other information it might need at a later time. My description here is a general one because I do not have a copy of their source code. I think it is unlikely that a walkthrough of their source code would reveal that it is possible to create a folder with a date which pre-dates the date when a new file is discovered.

by jeanmonroe 20.06.15 15:07

Nuala wrote:But Nuala has already said it's impossible so it must be impossible because Nuala knows exactly how the Way BAck machine program worked in 2007 and that's why he/she signed on urgently to tell us.

Still no real explantation from the real people.

Right, that's it, I'm gone.

I'm not putting up with such rudeness when all I'm doing is trying to help.

Just chill, Nuala.

You know how many people have been 'rude' to me over the years?

NONE!

They wouldn't 'dare'! laughat

I'm just getting to grips with 'Webscrumping', mentioned earlier.

Does that mean FREE 'virtual' apples?

by Joss 20.06.15 15:08

Richard D. Hall wrote:In order to understand what the WBM does, it is necessary to have a print out of their servers source code, which was running on 30 April 2007. This is something that WBM probably have archived somewhere and could look at. As far as I am aware nobody other than them have visibility of this code, therefore only they know exactly (and I mean EXACTLY) how their crawler functions. If you were to get the code and do a code walkthrough I suspect the code will show that their systems seeks out new webpages, and newly updated webpages. When it finds a new webpage (such as mccann.html) on the ceop website, it will invoke code to copy that page onto a folder on its server. In order to copy the new file it will have a function to create the new folder. This function I am sure will use the systems real time clock to construct the folder name based on the current time and day. Once the folder is made on their server, the file is then copied. Their system also ammends the file by adding a date and time stamp within the file and also some other information it might need at a later time. My description here is a general one because I do not have a copy of their source code. I think it is unlikely that a walkthrough of their source would reveal that it is possible to create a folder with a date which pre-dates the date when a new file is discovered.

That makes sense.

by MRNOODLES 20.06.15 15:09

jeanmonroe wrote:"This error is proving quite a hard task for us to explain at the moment"
------------------------------------------------------
OG.

"This 'abduction', of Madeleine, is proving quite a hard task for us to explain at the moment."

My money's on, after they've 'fixed it'. It'll lead to another question that some eagle eyed person will ask after spotting another anomaly.

by Joss 20.06.15 15:11

jeanmonroe wrote:
Nuala wrote:But Nuala has already said it's impossible so it must be impossible because Nuala knows exactly how the Way BAck machine program worked in 2007 and that's why he/she signed on urgently to tell us.

Still no real explantation from the real people.

Right, that's it, I'm gone.

I'm not putting up with such rudeness when all I'm doing is trying to help.

Just chill, Nuala.

You know how many people have been 'rude' to me over the years?

NONE!

They wouldn't 'dare'!

I'm just getting to grips with 'Webscrumping', mentioned earlier.

Does that mean FREE 'virtual' apples?

by Joss 20.06.15 15:17

Nuala wrote:But Nuala has already said it's impossible so it must be impossible because Nuala knows exactly how the Way BAck machine program worked in 2007 and that's why he/she signed on urgently to tell us.

Still no real explantation from the real people.

Right, that's it, I'm gone.

I'm not putting up with such rudeness when all I'm doing is trying to help.

It's ok. Nuala, i have enjoyed reading your posts & btw, welcome2

by SixMillionQuid 20.06.15 15:19

Just chiming in briefly.

1. If this was a 'glitch' why would an automated system pick a precise date time of 30/04/2007 11:58:03? It makes no sense to make up a date before the the web page became accessible to a webcrawler.

2. If this was a 'glitch' then other pages archived around the same time would show similar dates and times. Wayback should be able to confirm this.

by Joss 20.06.15 15:25

SixMillionQuid wrote:Just chiming in briefly.

1. If this was a 'glitch' why would an automated system pick a precise date time of 30/04/2007 11:58:03? It makes no sense to make up a date before the the web page became accessible to a webcrawler.

2. If this was a 'glitch' then other pages archived around the same time would show similar dates and times. Webback should be able to confirm this.

That would be good.

by Syn 20.06.15 15:31

Richard D. Hall wrote:In order to understand what the WBM does, it is necessary to have a print out of their servers source code, which was running on 30 April 2007. This is something that WBM probably have archived somewhere and could look at. As far as I am aware nobody other than them have visibility of this code, therefore only they know exactly (and I mean EXACTLY) how their crawler functions. If you were to get the code and do a code walkthrough I suspect the code will show that their systems seeks out new webpages, and newly updated webpages. When it finds a new webpage (such as mccann.html) on the ceop website, it will invoke code to copy that page onto a folder on its server. In order to copy the new file it will have a function to create the new folder. This function I am sure will use the systems real time clock to construct the folder name based on the current time and day. Once the folder is made on their server, the file is then copied. Their system also ammends the file by adding a date and time stamp within the file and also some other information it might need at a later time. My description here is a general one because I do not have a copy of their source code. I think it is unlikely that a walkthrough of their source code would reveal that it is possible to create a folder with a date which pre-dates the date when a new file is discovered.

/**
* Construct and initialize structure from a 14-digit String timestamp. If
* the argument is too short, or specifies an invalid timestamp, cleanup
* will be attempted to create the earliest legal timestamp given the input.
* @param dateStr from which to set date
*/
public Timestamp(final String dateStr) {
setDate(dateStrToDate(dateStr));

full source here:
[You must be registered and logged in to see this link.]

by jeanmonroe 20.06.15 15:37

Oh well, to add to my list of 'things' i 'don't get or understand', like, 'Chinese Arithmatic' and 'where do all those bullets, that people fire in the air, in 'celebration', above their heads, 'go', i can now add, 'WaybackMachine.'

Went to a restaurant and asked 'do you do take-aways'?

the guy said 'yes, of course'

'one minus one........nuffink'

by **Richard D. Hall** 20.06.15 15:48

Syn wrote:
Richard D. Hall wrote:In order to understand what the WBM does, it is necessary to have a print out of their servers source code, which was running on 30 April 2007. This is something that WBM probably have archived somewhere and could look at. As far as I am aware nobody other than them have visibility of this code, therefore only they know exactly (and I mean EXACTLY) how their crawler functions. If you were to get the code and do a code walkthrough I suspect the code will show that their systems seeks out new webpages, and newly updated webpages. When it finds a new webpage (such as mccann.html) on the ceop website, it will invoke code to copy that page onto a folder on its server. In order to copy the new file it will have a function to create the new folder. This function I am sure will use the systems real time clock to construct the folder name based on the current time and day. Once the folder is made on their server, the file is then copied. Their system also ammends the file by adding a date and time stamp within the file and also some other information it might need at a later time. My description here is a general one because I do not have a copy of their source code. I think it is unlikely that a walkthrough of their source code would reveal that it is possible to create a folder with a date which pre-dates the date when a new file is discovered.

/**
* Construct and initialize structure from a 14-digit String timestamp. If
* the argument is too short, or specifies an invalid timestamp, cleanup
* will be attempted to create the earliest legal timestamp given the input.
* @param dateStr from which to set date
*/
public Timestamp(final String dateStr) {
setDate(dateStrToDate(dateStr));

full source here:
[You must be registered and logged in to see this link.]

Is there an emulator I can run this code in?

by jeanmonroe 20.06.15 15:54

Oh, come 'back', Nullah.

Mrs Boss (me) will 'sing you to me'

"Somewhere, over the rainbow, waayyy up high"

We ain't all the 'same', be pretty damn 'boring' if we were, wouldn't it?

friends

----------------------------------------------------------------------------
"If it's a WaybackMachine error, then their credibility is screwed. But, if that date is accurate, it's dynamite!" (Almostgothic)

by Guest 20.06.15 16:05

Richard D. Hall wrote:In order to understand what the WBM does, it is necessary to have a print out of their servers source code, which was running on 30 April 2007. This is something that WBM probably have archived somewhere and could look at. As far as I am aware nobody other than them have visibility of this code, therefore only they know exactly (and I mean EXACTLY) how their crawler functions. If you were to get the code and do a code walkthrough I suspect the code will show that their systems seeks out new webpages, and newly updated webpages. When it finds a new webpage (such as mccann.html) on the ceop website, it will invoke code to copy that page onto a folder on its server. In order to copy the new file it will have a function to create the new folder. This function I am sure will use the systems real time clock to construct the folder name based on the current time and day. Once the folder is made on their server, the file is then copied. Their system also ammends the file by adding a date and time stamp within the file and also some other information it might need at a later time. My description here is a general one because I do not have a copy of their source code. I think it is unlikely that a walkthrough of their source code would reveal that it is possible to create a folder with a date which pre-dates the date when a new file is discovered.

Exactly.

I don't like "new" members with a mission to educate us on a VERY hot topic telling us that they know it's impossible because well... they say so.

So excuse my Yorkshire bluntness, why mess about?

by sallypelt 20.06.15 16:07

BlueBag wrote:
Richard D. Hall wrote:In order to understand what the WBM does, it is necessary to have a print out of their servers source code, which was running on 30 April 2007. This is something that WBM probably have archived somewhere and could look at. As far as I am aware nobody other than them have visibility of this code, therefore only they know exactly (and I mean EXACTLY) how their crawler functions. If you were to get the code and do a code walkthrough I suspect the code will show that their systems seeks out new webpages, and newly updated webpages. When it finds a new webpage (such as mccann.html) on the ceop website, it will invoke code to copy that page onto a folder on its server. In order to copy the new file it will have a function to create the new folder. This function I am sure will use the systems real time clock to construct the folder name based on the current time and day. Once the folder is made on their server, the file is then copied. Their system also ammends the file by adding a date and time stamp within the file and also some other information it might need at a later time. My description here is a general one because I do not have a copy of their source code. I think it is unlikely that a walkthrough of their source code would reveal that it is possible to create a folder with a date which pre-dates the date when a new file is discovered.
Exactly.

I don't like "new" members with a mission to educate us on a VERY hot topic telling us that they know it's impossible because well... they say so.

So excuse my Yorkshire bluntness, why mess about?

Sleeping spiders, spider crawls, webscraping...............it's all getting too much for me................HELP!!!!

by Guest 20.06.15 16:08

SixMillionQuid wrote:Just chiming in briefly.

1. If this was a 'glitch' why would an automated system pick a precise date time of 30/04/2007 11:58:03? It makes no sense to make up a date before the the web page became accessible to a webcrawler.

2. If this was a 'glitch' then other pages archived around the same time would show similar dates and times. Wayback should be able to confirm this.

Yes - and provide examples.

A point I made hundreds of posts ago.

by sallypelt 20.06.15 16:10

BlueBag wrote:
SixMillionQuid wrote:Just chiming in briefly.

1. If this was a 'glitch' why would an automated system pick a precise date time of 30/04/2007 11:58:03? It makes no sense to make up a date before the the web page became accessible to a webcrawler.

2. If this was a 'glitch' then other pages archived around the same time would show similar dates and times. Wayback should be able to confirm this.
Yes - and provide examples.

A point I made hundreds of posts ago.

If what my limited IT knowledge is telling me, then what I am understanding is, if a person or company is designing a web page, there are programmes available which can capture that information WITHOUT it having hypertext before it goes live. Please techies, can you tell me that I am barking up the wrong tree?

by Guest 20.06.15 16:13

Syn wrote:
Richard D. Hall wrote:In order to understand what the WBM does, it is necessary to have a print out of their servers source code, which was running on 30 April 2007. This is something that WBM probably have archived somewhere and could look at. As far as I am aware nobody other than them have visibility of this code, therefore only they know exactly (and I mean EXACTLY) how their crawler functions. If you were to get the code and do a code walkthrough I suspect the code will show that their systems seeks out new webpages, and newly updated webpages. When it finds a new webpage (such as mccann.html) on the ceop website, it will invoke code to copy that page onto a folder on its server. In order to copy the new file it will have a function to create the new folder. This function I am sure will use the systems real time clock to construct the folder name based on the current time and day. Once the folder is made on their server, the file is then copied. Their system also ammends the file by adding a date and time stamp within the file and also some other information it might need at a later time. My description here is a general one because I do not have a copy of their source code. I think it is unlikely that a walkthrough of their source code would reveal that it is possible to create a folder with a date which pre-dates the date when a new file is discovered.

/**
* Construct and initialize structure from a 14-digit String timestamp. If
* the argument is too short, or specifies an invalid timestamp, cleanup
* will be attempted to create the earliest legal timestamp given the input.
* @param dateStr from which to set date
*/
public Timestamp(final String dateStr) {
setDate(dateStrToDate(dateStr));

full source here:
[You must be registered and logged in to see this link.]

I'm a professional java programmer with 15 years java experience.

You've told us nothing here except what the method comment says.

How does this relate to the issue at hand?

Log in for more topics