Best Software to Track when a link I have Submitted goes live

[Firestarter-SEO]

Let's say I built a directory link and it takes a while to go live. Is there a tool that will crawl the site and notify me when the link goes live?

[effectdigital]

There are several ways to do this but it does depend somewhat on the architecture of the site which is hosting the link you have placed. For example, you will want to navigate around a bit to see the difference between links in review and those which have gone live. Quite commonly you will submit a link and get a temporary URL like https://somelocaldirectory.com/review/my-clients-company.

Once the link goes live the address can sometimes actually change to something more like https://somelocaldirectory.com/listings/us/california/automotive/insurance/my-clients-company. If you can work out how the architecture changes, you can simply build a list of the final URLs that 'should' exist (but don't yet) and you can query them daily to determine when the link goes live (most likely when it returns status code 200 / OK). Note that for this method your URL syntax needs to be near perfect as, for example - if you end the URL string with a slash ("/") and the site uses 301 redirects to strip all trialling slashes, you will likely see the query turn from a 404 to a 301 instead of from a 404 to a 200 (so some common sense is needed).

You can build a spreadsheet with all your 'presumed' live links in and every day, run a VBA macro or use an Excel Add-In to return the status codes of the URLs housed within your query-column. You could install and utilise a plugin like SEO tools for Excel so that Excel's native formulae functionality is extended to return status codes (meaning you can write formulae such as "=HttpStatus(A1)"). The creator (Nimbasa I believe) has now made the plugin premium, but if you perform some advanced web searches you can still find the old version(s) from when the plugin was completely free. That version of the plugin (4.0.7 I believe) is fully compatible with Excel 2016 (32 or 64 bit) and it does have the status code checking function under the SEOTools->Onpage->HttpStatus menu entry.

If you can produce such a list of URLs you could also bulk query that list using Screaming Frog SEO Spider's list mode and just take the status codes from there. You could also write XPath expressions to check not only when those pages go live, but whether they are linking to your client's site, citing the URL in an unlinked way as well as detecting the nature of the link.

If all of that sounds like a lot of effort, I have found that running a 'backlink project' (not just checking via the front-end, setting up a backlink project for a specific site!) in Ahrefs seems to find most new links of reasonable value. It does rely on Ahrefs having indexed the site / URL in question, but as long as the site isn't pure spammy garbage their hyperlink indexation is really good. You can set up alerts via email to notify you of newly detected links! Once some new links are detected, you can check to see whether your assumed link is among those which was recently set live and then indexed by Ahrefs. This is much easier but is less accurate and usually your notifications of live links are a bit slower (3-10 days after the link 'really' goes live, Ahrefs usually finds it). You need an Ahrefs subscription, but it's a great tool - useful for many things.

For a solution that's quick, accurate and minimal hassle you really want to look at "Monitor Backlinks". There used to be free, downloadable tools which did pretty much the same thing.

One more flexible tool was the old freeware version of WebMon. It's a great tool for entering URLs and then receiving notifications when the HTML on a specified area of the page (could be the whole page, or just one link - etc) changes. Unfortunately the freeware version of the tool hasn't been updated in many years and for some reason it cannot support HTTPS URLs. This was no big deal at the time, but with most of the 'well-known' web shifting over to HTTPS, obviously this severely dampens the tool's utility.

Even if you don't want to purchase and utilise a tool like ScrapeBox, you may be interested to know that they released the handy (and free)ScrapeBox Link Checker. It's really very useful and may be just what you are after, but again it hinges on you knowing the URL (or URLs, plural) which your page / site is supposed to be receiving a hyperlink from. This tool is pretty neat and I end up using it now and then. To be fair you can do pretty much the same thing with Screaming Frog but it takes more XPath and / or Regex knowledge. I tend to use both as I find each tool finds some links that the other tool does not find (thus increasing my overall data integrity when checking thousands of backlinks for their live status).

Although it's primarily a link 'discovery' tool (much like Moz's Link Explorer), you can actually use Link Assistant's Power Suite SEO SpyGlass for this kind of work. You just start a project, once SEOSG pulls back some links - delete them all from the project. Then, import your own links and run the 'checker' which will query the specified URLs for links to your client's site. This one is pretty robust but it takes as few minutes to set up the project correctly and it's not extremely resource efficient. Again - I like to aggregate it with results from other tools to get a clearer, more realistic picture of who is (and is not) linking back.

All of these tools are great but they all work in different ways. Some will alert you to any new links to your site at the domain level (like Ahrefs) but rely on their own crawlers and indexation to keep up. Others can very quickly tell you exactly what's going on, but require being set up in a binary way - possibly with some coding know-how.

Try a few, see what you like. See what works for you!

Hope that helps

[Saijo.George]

That will depend on the directory tools like Synup, Moz Local can tell you when they go up on certain sites.

If it's not a popular one there are a few different scenarios :

1. Do they give you a temporary page that is only visible to you and will go live after it's approved? You can check the HTTP header response fo it and if it returns 200 it's probably approved and live.
2. Do they give you a live URL that says it's pending verification? Scrape the page and use XPath to see id that bit of text changes.
3. etc

Give us some more info on it and perhaps we can help.