Issues with feeds

Hello Tim,

For the Rakuten feed, use the Parse Analysis tool (Tools > Feed Utilities) to see which of the required fields are being reported as invalid and then double check on Feed Registration Step 2 that the correct field has been mapped. It's most common I have found for the price field to be invalid, and what can happen with XML feeds is where the price is actually, for example PRICE/GBP. In this case, the auto-detection would have picked "PRICE", which would be empty for all products - that may be all it is...

For the Clixgalore feed; it may the case where there is a repeating element within the product records that has fooled the format auto-detection. For example, if you see something like this on Feed Registration Step 1:

...this would need to be edited down to just:

In either case, if you're still not sure if you're happy to post the links (I'll remove before publishing your reply) I'll check them out on my test server for you...

Cheers,
David.
--
PriceTapestry.com

Ah great, thanks. The Parse Tool worked great, except the results are a bit strange. I imported them once and the total records were around 126, then again and they changed to 122 without any other changes.

This is the feed in question.

{link saved}

It's importing products now which is great, however, I can see that a particular product is in the feed when I view in an HTML editor, but when I try and map it, it doesn't come up in the search results.

On a similar note, product mapping seems to be hit and miss for me, unless I'm not inputting things correctly. I'll do a search and then add all of the relevant terms into the box on the left. Often though the results on the frontend will only show 1 or 2 products. I even tried entering one term that would definitely match a lot of products (without an equals sign) and it didn't return any products on the frontend. I'm probably doing something wrong! :$

The ClixGalore feed is:

{link saved}

This is the one I can't seem to get to work with the registration/field mapping.

Any help, much appreciated! :)

Hi Tim,

Regarding the Rakuten feed, I wasn't able to fetch the URL as it returned an FTP error "550 No Such Direction" - could you check?

In the mean time I'll continue to look into the Clixgalore feed - it doesn't appear to contain any line endings so you may need to go back to technical support at the network but I'll let you know...

Cheers,
David.
--
PriceTapestry.com

Thanks David.

Can you try this one for Rakuten: {link saved}

[If not I can email it to you, but I've tested and it's working.]

I've switched to using their CSV/TXT files to see if it helps, but now I'm getting the issue where the options in the registration mapper are all 'FIELD1, FIELD2 etc'.

Using this setting as it was auto-populated: csv|124|0|0|1

Rakuten does give the following advice in their documentation which may be affecting things but I'm not sure how to check or implement:

NOTE: If you are using a script to retrieve the files, wget is preferred over curl. If using curl, you must use the option --ignore-content-length.

To make things more complicated, and it may well be linked to the above 'note', but when I try to import a fairly large 25mb feed from them, it seems to fail to bring in any products properly via slow import. 'Import' seems to do something if I use that option and then click on slow import afterwards, it shows as DELETING. Then I'll manually click back to the home admin screen and it shows the 200k+ products. However, when I use the shortcode to display the price table with 'currency="USD"', it doesn't show anything which makes me think the import didn't quite finish or something?

This is the large USD feed: {link saved}

Hi Tim,

The CSV version worked fine (using CURL without having to make any changes to ignore content length). The auto-detected format string is correct - the script checks for "HDR" at the start and adds the additional "|1" to the format string which makes the parser skip that line. I then registered the required fields as;

Product Name: FIELD2
Buy URL: FIELD6
Price: FIELD14

..and the feed imported 3542 products. Can you try the same and confirm if the products are imported and then let me know if the issue regarding Product Mapping remains and I'll check that out further with you.

Regarding the large XML feed, unfortunately the slow import method is not ideal for XML feeds and I am actually considering discontinuing slow import support for XML - the problem is being able to precisely identify the position in the stream as this is reported differently depending on the XML library incorporated into the PHP build. Ideally, large feeds should be imported from the command line. If you have command line access to your account, the process is simply to change directory to the scripts folder e.g.

cd public_html/scripts/

...and then run fetch.php with the script filename.e.g

php import.php filename.xml

Cheers,
David.
--
PriceTapestry.com

Thanks so much, David. The feeds seem to be better on CSV generally.

Any ideas on that ClixGalore feed or should I contact the network?

Hi Tim,

The line ending of the ClicGalore feed doesn't seem to be compatible with PHP's fgetcsv() function which is used in the "Faster CSV" version of the parser distributed with Price Tapestry.

However the full version (which includes multi-line record support) works fine, so if you could please email me I'll forward the full version for you to try...

Cheers,
David.
--
PriceTapestry.com

Ok thanks, I've emailed you now.

Tim