Twitter SMFHacks Facebook SMFHacks SMFHacks.com
** Home Forum Index Hacks Products Login Register Search
Welcome, Guest. Please login or register.
September 01, 2014, 08:47:56 pm

Login with username, password and session length
Members
Total Members: 11338
Latest: maxx123
Stats
Total Posts: 35402
Total Topics: 5928
Online Today: 85
Online Ever: 2482
(April 09, 2011, 07:02:45 pm)
Users Online
Users: 1
Guests: 81
Total: 82
+ 
|-+ 
| |-+ 
| | |-+ 
| | | |-+ 
0 Members and 1 Guest are viewing this topic. « previous next »
Pages: [1] Go Down Print
Author Topic: Blocking spiders and search engines from downloading our service manuals files?  (Read 1704 times)
ApplianceJunk
Sr. Member
****
Offline Offline

Posts: 414



View Profile WWW
« on: January 28, 2012, 03:38:51 pm »

When I open up our download section for guest to use I take it that spiders and such are also able to click the download button and I'm concerned that will increase our monthly band width usage. Is that correct?

I have put a daily limit of three downloads for guest and a wait time of two minutes between downloads, but is there anything I can do to keep spiders and such from downloading our files?

I want google and such to be able to view the download page yet, but just not download the .pdf files we have on them.

What do you think?

Thanks,
Logged

ApplianceJunk
Sr. Member
****
Offline Offline

Posts: 414



View Profile WWW
« Reply #1 on: January 29, 2012, 11:46:21 am »

Any thoughts on this?

Even though spiders can click on the download button do they really download a file to anyplace after that?

Is there any reason to even be concerned about spiders increasing our band width usage by them downloading our .pdf files?

If spiders can download our .pdf files from our download section how about adding a recaptcha or spam verification question that has to be filled in before the download works for guest?

Logged

SMFHacks
Administrator
Hero Member
*****
Offline Offline

Posts: 12109


View Profile
« Reply #2 on: January 29, 2012, 01:00:05 pm »

I wouldn't be worried about the bandwidth usage they should only hit the files one time to index them.

I could add some code from blocking to access that page as an option though.
Logged

Get your Forum Ranked! at http://www.forumrankings.net - find out how your forum compares with others! Also contests/promotions for verified sites.
ApplianceJunk
Sr. Member
****
Offline Offline

Posts: 414



View Profile WWW
« Reply #3 on: January 29, 2012, 01:07:58 pm »

I don't want to block them all together from viewing the download page, so I will just keep a eye on it and see how it goes.
If they only hit it once I guess it could be a good thing for use if we end up getting a few more URL's indexed by it.

Having to purchase more bandwidth would not be the worst thing in the world either. Wink

One of our affiliates we work with wanted us to open our downloads so you did not have to register to download a single service manual or two so I'm giving it a try.

We did not have as many new members register yesterday for our site and I believe this is because we opened up our downloads to guest like our affiliate wanted, but we had a great day for selling parts.

Thanks for the reply.
Logged

SMFHacks
Administrator
Hero Member
*****
Offline Offline

Posts: 12109


View Profile
« Reply #4 on: January 29, 2012, 03:40:59 pm »

Made an update that may help
2.1.1
+Added setting to deny access to search engines to download file to prevent indexing/resource usage.
+Added setting for basic hotlink protection
Logged

Get your Forum Ranked! at http://www.forumrankings.net - find out how your forum compares with others! Also contests/promotions for verified sites.
ApplianceJunk
Sr. Member
****
Offline Offline

Posts: 414



View Profile WWW
« Reply #5 on: January 29, 2012, 03:48:34 pm »

Made an update that may help
2.1.1
+Added setting to deny access to search engines to download file to prevent indexing/resource usage.
+Added setting for basic hotlink protection


Awesome, Thanks!
I'm currently using 2.0.4.
Do I need to uninstall it first?
Logged

SMFHacks
Administrator
Hero Member
*****
Offline Offline

Posts: 12109


View Profile
« Reply #6 on: January 29, 2012, 03:54:12 pm »

I suggest an uninstall  if possible when going to 2.1 since a new tab is added in the Admin.php file
Logged

Get your Forum Ranked! at http://www.forumrankings.net - find out how your forum compares with others! Also contests/promotions for verified sites.
ApplianceJunk
Sr. Member
****
Offline Offline

Posts: 414



View Profile WWW
« Reply #7 on: January 29, 2012, 04:01:46 pm »

I suggest an uninstall  if possible when going to 2.1 since a new tab is added in the Admin.php file

Ok, I just tried to but was shown all these failed.

Hope I don't have to do a lot of manual editing...
Logged

SMFHacks
Administrator
Hero Member
*****
Offline Offline

Posts: 12109


View Profile
« Reply #8 on: January 29, 2012, 04:04:03 pm »

I would skip that part then just install over the top and make the changes to Admin.php if they are not down yet.
Logged

Get your Forum Ranked! at http://www.forumrankings.net - find out how your forum compares with others! Also contests/promotions for verified sites.
ApplianceJunk
Sr. Member
****
Offline Offline

Posts: 414



View Profile WWW
« Reply #9 on: January 29, 2012, 04:07:25 pm »

I would skip that part then just install over the top and make the changes to Admin.php if they are not down yet.

Ok, so when you say 'skip that part' you mean do not uninstall 2.0.4, just upload and install 2.1.1

Thanks,
Logged

SMFHacks
Administrator
Hero Member
*****
Offline Offline

Posts: 12109


View Profile
« Reply #10 on: January 29, 2012, 04:18:09 pm »

Yes just install the latest version.
Logged

Get your Forum Ranked! at http://www.forumrankings.net - find out how your forum compares with others! Also contests/promotions for verified sites.
Pages: [1] Go Up Print 
« previous next »
Jump to:  

Recent
[Today at 03:04:38 pm]

[Today at 09:03:35 am]

[August 31, 2014, 09:40:44 pm]

[August 31, 2014, 05:06:49 pm]

[August 30, 2014, 10:02:54 pm]

[August 30, 2014, 01:12:42 pm]

[August 29, 2014, 02:23:21 pm]

[August 28, 2014, 11:01:47 pm]

[August 28, 2014, 08:13:15 pm]

[August 27, 2014, 10:08:34 pm]
Random Picture
Donate to SMFHacks.com
Help Support the SMFHacks.com mod making.
Powered by SMF 1.1.19 | SMF © 2013, Simple Machines
TinyPortal v0.9.7 © Bloc
SMF and SimpleMachines are registered trademarks of Simple Machines. SMFHacks.com is not affiliated with nor endorsed by Simple Machines.
Page created in 1.17 seconds with 18 queries.