Legacy Add the ability to include a separate list of urls outside of vb standard urls

[bzcomputers]

I personally have quite a few add-on pages that will not currently be picked up by the DBSEO Sitemap feature. This includes personally created custom pages and some pages that are created by additional add-ons to vb4.

This was something that was handled previously by vbSEO's sitemap "extra-urls" feature.

I looked into doing it myself and actually started coding something but seemed to not be able to make much progress trying to tap into the DBSEO sitemap side or directly into the vb sitemap. Even if I was able to finish it (which I'm not certain I could anyway ), I'd hate for it not to work/break in the future.

What I was looking to do was just mimic the old vbSEO option which was to just recognize the existence of the standard named .txt file and then import the urls into the sitemap all with a weight of "1.0". Which is all I would personally need and probably all that anyone else would need also.

You could build further on this by adding option(s):

1) Allowing users to define different names for additional url lists (.txt files)
2) Then be able to assign different weights to each different url list (.txt file)
...but that may be overkill​

 

[CharlieDelta]

I to have some custom pages I made myself and would take advantage of something like this.

 

[Fillip H.]

Since I'm not entirely sure about the use cases for this functionality, I could approach it one of two ways:

Either I could have a text field like the Custom Rewrite Rules and Custom 301 Redirects settings, or I could create an interface for adding URLs that would have similar configuration options like the current Sitemap settings for each content.

The benefit of an interface is that you'll have fine grained control over the Last Update Date, Priority and Update Frequency, but if we're talking hundreds of URLs it could be cumbersome to add them all.

The benefit of a simple field is that it's very easy to edit.

How do you guys intend to use the functionality cruisin and CharlieDelta ? Dozens / hundreds of URLs or few enough that a multi-add interface is viable?

 

[bzcomputers]

I personally have hundreds. The best way for me would be to use a separate .txt file with each "extra-url" on a separate line within the file.

I know this will limit the control over Last Update Date, Priority and Update Frequency.

What I was doing prior was working well in search rankings and that was to use a priority of "1.0", a Last Modified/Update Date as todays date (the current date each time the sitemap is created), and a Change Frequency of daily across the board for all urls within the file.

 

[Fillip H.]

Was the format for the extra-urls.txt file just a simple "one URL per line" file? If so, I could create an importer into an interface. You'd be able to edit the aforementioned options post-import, should you want to.

Would that work?

 

[bzcomputers]

Was the format for the extra-urls.txt file just a simple "one URL per line" file? If so, I could create an importer into an interface. You'd be able to edit the aforementioned options post-import, should you want to.

Would that work?

Yes, it was just a one URL per line file with no other data in the file.

Not sure I followed what you were saying after that, so I'll just throw out what I was thinking and you can let me know if we're on the same page.

I personally add new links to the list often, more often than I run the sitemap. I'd like the sitemap to just automatically re-import the list at each sitemap run.

Then in the settings what might be best would be to have another main option under "Sitemap Settings" called "Extra-urls" with these settings underneath it:

1) Do you have an "extra-urls" file?
2) Extra-urls Priority?
3) Extra-urls Update Frequency?

 

[CharlieDelta]

How do you guys intend to use the functionality cruisin and CharlieDelta ? Dozens / hundreds of URLs or few enough that a multi-add interface is viable?

I only have a few.

 

[Fillip H.]

cruisin This is the interface I threw together, which would give you an idea of how it could work: https://www.dropbox.com/s/d2lrmb58qkho2oj/Screenshot 2013-12-10 21.24.07.png

I'm not sure how well the mass update interface would hold up, considering Suhosin and PHP 5.3.9+ have restrictions on the maximum number of form elements. I may have to provide an alternate interface if too many form elements are detected, though that's tomorrow's problem.

Let me know your thoughts

 

[bzcomputers]

cruisin This is the interface I threw together, which would give you an idea of how it could work: https://www.dropbox.com/s/d2lrmb58qkho2oj/Screenshot 2013-12-10 21.24.07.png

I'm not sure how well the mass update interface would hold up, considering Suhosin and PHP 5.3.9+ have restrictions on the maximum number of form elements. I may have to provide an alternate interface if too many form elements are detected, though that's tomorrow's problem.

Let me know your thoughts

That looks like a ton of work to code!

From looking at this current setup it appears that the import of the custom urls is a manual process each time? I was hoping that this would automated by using either a set location/filename (or set by user option) that if it exists the file will automatically be imported on each sitemap creation.

I also see you are creating a separate sitemap with these urls. For keeping it simple I know what I was using previously just appended the imported urls contained within the file to the beginning of the Forum URLs sitemap. Not sure if there are any pros and cons as far as SEO goes, but it would probably be easier as far as coding goes.

 

[Fillip H.]

Is it not the same amount of work to import a text file into that interface as it is to open your FTP app -> navigate to the file in question -> edit -> save?

I'm not sure I see the benefit of reading from a file.

 

[bzcomputers]

Is it not the same amount of work to import a text file into that interface as it is to open your FTP app -> navigate to the file in question -> edit -> save?

I'm not sure I see the benefit of reading from a file.

I personally create new content files typically on a regular basis sometimes a few a day, although sometimes none for a week or better. After each new page is created I save any new urls to the extra-urls file, knowing that when the automated sitemap runs either that day or within the next couple of days they'll automatically be included in the latest sitemap.

I guess I've just been use to thinking that when the automated sitemap runs whether I'm around or not, I know all the latest urls have been added and submitted.

I can definitely make either way work and the simpler the better in my opinion.

 

[Mick]

Fillip H.,

That Looks AMAZING mate.

So impressed with the work you are doing on the SEO front, the interface is perfect it will be brilliant for custom pages!!!

Thanks
Mick