Error message while walking/indexing a site

[turtle]

I received the following error messages while a test site was being walked.

[Admin]

Hi and thanks for the test!
'Referential Integrity Violation' error, as well as 'Cannot perform this operation...' error will be fixed in the next 1.3 version of Site Visualizer.

As for the 'Source string is too wide...' warning message, this means that the meta keywords text length of the page specified is 2770 bytes (1385 characters), and it's too wide to be stored in the database field that is bounded by 2048 bytes (or 1024 characters). What is your mind on how to do better: crop such too wide meta keyword tags to 1024 characters, or extend the DB field?

[turtle]

It appears that the errors are on pages that do not exist (improper syntax for the URL construction within the error message).

A little insight in how the software walks the site might be helpful in pin pointing where the walked item URL is getting goofy.

The site I first tested WSV on is a full dynamic site (i.e. no html pages exist for the domain). The domain's pages are constructed "on the fly" from a database from visitor/user requests (menu/link selections out from current page). I chose a full dynamic site as I thought this might be the most difficult to walk as there is nothing to walk until a link/menu item were followed.

The domain is not mine but a customer's. I've looked into the database and the field that is throwing the "string too long" error (over 2048 characters) has only 255 characters maximum allowed so even if the record that is triggering the error was not "trimmed" at the time it was saved the max the field could be would only be 255 characters long, even with untrimmed leading and trailing spaces



I don't think that the error that is being triggered is a field length issue but something else.

It might be a good idea to truncate/trim the string you are using to the max allowed for your database to take into consideration dynamic sites that are not constructed with proper forethought (someone lets all fields in their table use the DB max default length)

Where does WSV store its database info? I cannot find anything in the install folder that looks like a database (not even a CSV file).

[Admin]

I'm not sure on how does meta keywords tag is being generated on the server side of the site, but here is an example of such "too wide" text (1385 symbols):

Code: Select all

<meta name="keywords" content="bronze scupture, cast bronze, animalier, sporting art, french bronzes, antoine louis barye, alfred barye, andre vincent becquerel, isidore bonheur, rembrandt bugatti, auguste cain, victor chemin, georges collard, renzo colombo, paul comolera, paul delabrierre, christophe fratin, emmanuel fremiet, georges gardet, gaston d'illers, prosper lecourtier, pierre lenordez, max leverrier, clovis masson, pierre jules mene, jules moigniez, louis de monard, charles paillet, paul paulin, francois pompon, ferdinand pautrot, auguste rodin, louis riche, goido righetti, edouard-marcel sandoz, pierre nicholas tourgeneff, charles valton, louis vidal, jasper d'ambrosi, bryant baker, paul wayland bartlett, ruth brooks, edwin codman, bunny connell, cyrus dallin, edwin deming, laura gardin fraser, james earl fraser, harriet frishmuth, eli harvey, malvina hoffman, charles humphriss, anna hyatt huntington, bremner jackson, max kalish, t.d.kelsey, edward kemeys, marguerite kermse, isidore konti, robert tait mckenzie, james wilson macdonald, frederick macmonnies, carol brooks macneil, paul manship, william clark noble, alexander Phimister proctor, frederic remington, frederick roth, charles marion russell, augustus saint-gaudens, jannet scudder, joy clinton shepherd, john quincy adams ward, katherine weems, hughlette wheeler, kathleen wheeler, lila wheelock, mahonri young, federick ziegler" />


It appears on the home page of the site you've tested on: http://bronze-gallery.com/

Regarding to the internal Site Visualizer's database, it's stored in the *.SVZ file and is temporary unpacking to the app's Application Data directory, e.g.: C:\Users\oleg\AppData\Roaming\Site Visualizer\
But I'd not recommend you to use the database explicitly - use the Database tab instead.

[turtle]

The solution to the too long would be to truncate when you save your record to the max you allow, rather than modify the software to accommodate something that it so long. That keyword list is obviously too long. Maybe a note within the results that one of the items is inappropriately too long and has been truncated.

The initial index.cfm page did not have the meta tags built from the database content so it was outside of the MS SQL server database (i.e. the keywords were manually added to the static page at the time it was created rather than having the database populate the meta tag.

Editing down to under the 1024 limit then re-walking showed a 2nd page with this same long list which I also edited.



Site walks without errors now that there are no pages that accede your storage limit..

I believe that truncating your database save to the max allowed in your stored table would be the solution. You cannot accommodate bad web design and coding but you can "ignore it" when you come across it

[Admin]

That keyword list is obviously too long

This is one of Site Visualizer's aims: to reveal a website's internal structure errors

You cannot accommodate bad web design and coding

Exactly!
Maybe in future we'll implement a feature allowing to set a fields' length in the DB, but it's definitely not a urgent task.