As you may know or not ! Magento is the fastest growing eCommerce plateform, nowonder because Magento has a lots of flexibility in it. You can feel that too if you are a just a user and as for me as Programmer I just Love it. One of the salient feature that makes Magento a leader in eCommerce software is its ability to Import and Export data to an from the system.
Magento gives you a default interface to import or export data, but you will need the specific format of your data.It is easy to customize that as well, but today lets be interested in standard import, but with one small (Yet important) modification. The important modification in the standard code is this code will be able to options to the attribute of Select/Multiselect/Radio/Checkbox input type, while the product is being imported.
By default Magento only imports the values of the option, of any attribute, which are already avaliable. But this code can be used to create option at run time if it does not exits and assign to the product, during import. For doing this we need to first set up an Advanced Profile of Export/Import. Go to System–>Import/Export–>Advanced Profile and create new Profile. Then in the Action XML paste this.
<!--
Path=Path of the file relative to ROOT
filename= File name of the XML(MS EXCEL 2003) where import data is saved in Magento's Format
format = XML
You can use CSV as well.
-->
<action type="dataflow/convert_adapter_io" method="load">
<var name="type">file</var>
<var name="path">var/import</var>
<var name="filename"><![CDATA[products.xml]]></var>
<var name="format"><![CDATA1]></var>
</action>
<!--
adapter= Your Model path
-->
<action type="dataflow/convert_parser_xml_excel" method="parse">
<var name="single_sheet"><![CDATA[]]></var>
<var name="fieldnames">true</var>
<var name="store"><![CDATA[0]]></var>
<var name="number_of_records">1</var>
<var name="decimal_separator"><![CDATA[.]]></var>
<var name="adapter">yourmodel/import</var>
<var name="method">save</var>
</action>
The Model defined in the adapter should be like this. You can create a new module for this.
<?php
class Namespace_Module_Model_Import extends Mage_Catalog_Model_Convert_Adapter_Product {
/**
* Save product (import)
* @param array $importData
* @throws Mage_Core_Exception
* @return bool
*/
public function saveRow(array $importData)
{
$product = $this->getProductModel()
->reset();
if (empty($importData['store'])) {
if (!is_null($this->getBatchParams('store'))) {
$store = $this->getStoreById($this->getBatchParams('store'));
} else {
$message = Mage::helper('catalog')->__('Skip import row, required field "%s" not defined', 'store');
Mage::throwException($message);
}
}
else {
$store = $this->getStoreByCode($importData['store']);
}
if ($store === false) {
$message = Mage::helper('catalog')->__('Skip import row, store "%s" field not exists', $importData['store']);
Mage::throwException($message);
}
if (empty($importData['sku'])) {
$message = Mage::helper('catalog')->__('Skip import row, required field "%s" not defined', 'sku');
Mage::throwException($message);
}
$product->setStoreId($store->getId());
$productId = $product->getIdBySku($importData['sku']);
if ($productId) {
$product->load($productId);
}
else {
$productTypes = $this->getProductTypes();
$productAttributeSets = $this->getProductAttributeSets();
/**
* Check product define type
*/
if (empty($importData['type']) || !isset($productTypes[strtolower($importData['type'])])) {
$value = isset($importData['type']) ? $importData['type'] : '';
$message = Mage::helper('catalog')->__('Skip import row, is not valid value "%s" for field "%s"', $value, 'type');
Mage::throwException($message);
}
$product->setTypeId($productTypes[strtolower($importData['type'])]);
/**
* Check product define attribute set
*/
if (empty($importData['attribute_set']) || !isset($productAttributeSets[$importData['attribute_set']])) {
$value = isset($importData['attribute_set']) ? $importData['attribute_set'] : '';
$message = Mage::helper('catalog')->__('Skip import row, is not valid value "%s" for field "%s"', $value, 'attribute_set');
Mage::throwException($message);
}
$product->setAttributeSetId($productAttributeSets[$importData['attribute_set']]);
foreach ($this->_requiredFields as $field) {
$attribute = $this->getAttribute($field);
if (!isset($importData[$field]) &amp;amp;&amp;amp; $attribute &amp;amp;&amp;amp; $attribute->getIsRequired()) {
$message = Mage::helper('catalog')->__('Skip import row, required field "%s" for new products not defined', $field);
Mage::throwException($message);
}
}
}
$this->setProductTypeInstance($product);
if (isset($importData['category_ids'])) {
$product->setCategoryIds($importData['category_ids']);
}
foreach ($this->_ignoreFields as $field) {
if (isset($importData[$field])) {
unset($importData[$field]);
}
}
if ($store->getId() != 0) {
$websiteIds = $product->getWebsiteIds();
if (!is_array($websiteIds)) {
$websiteIds = array();
}
if (!in_array($store->getWebsiteId(), $websiteIds)) {
$websiteIds[] = $store->getWebsiteId();
}
$product->setWebsiteIds($websiteIds);
}
if (isset($importData['websites'])) {
$websiteIds = $product->getWebsiteIds();
if (!is_array($websiteIds)) {
$websiteIds = array();
}
$websiteCodes = split(',', $importData['websites']);
foreach ($websiteCodes as $websiteCode) {
try {
$website = Mage::app()->getWebsite(trim($websiteCode));
if (!in_array($website->getId(), $websiteIds)) {
$websiteIds[] = $website->getId();
}
}
catch (Exception $e) {}
}
$product->setWebsiteIds($websiteIds);
unset($websiteIds);
}
foreach ($importData as $field => $value) {
if (in_array($field, $this->_inventoryFields)) {
continue;
}
if (in_array($field, $this->_imageFields)) {
continue;
}
$attribute = $this->getAttribute($field);
if (!$attribute) {
continue;
}
$isArray = false;
$setValue = $value;
if ($attribute->getFrontendInput() == 'multiselect') {
$value = split(self::MULTI_DELIMITER, $value);
$isArray = true;
$setValue = array();
}
if ($value &amp;amp;&amp;amp; $attribute->getBackendType() == 'decimal') {
$setValue = $this->getNumber($value);
}
/**CODE MODIFICATION STARTS HERE*/
$optionLabelArray=array();
if ($attribute->usesSource()) {
$options = $attribute->getSource()->getAllOptions(false);
/**
* Update the Source of the attribute when source has no options.
*/
if(count($options)<1){
if($isArray){
foreach($value as $key=>$subvalue){
if(!in_array($subvalue,$newOptionLabelArray)){
$setValue[]=$this->updateSourceAndReturnId($field,$subvalue);
array_push($newOptionLabelArray,$subvalue);
}
}
}
else{
if(!in_array($value,$newOptionLabelArray)){
$setValue=$this->updateSourceAndReturnId($field,$value);
array_push($newOptionLabelArray,$value);
}
}
}
/**
* Work on the source when it has options
*/
else{
/**
* This is the case of Multi-Select
*/
if ($isArray) {
foreach ($options as $item) {
/** Setting the option's ID if Label matches with the current value of XML column.*/
if (in_array($item['label'], $value)) {
$setValue[] = trim($item['value']);
array_push($optionLabelArray,$item['label']); /**Adding Reference to worked attribute option**/
}
}
/**
* Checking in the current XML column value if all values were used in the above loop or not
* If not used then they are new options value, then new option is created and then assigned.
**/
foreach($value as $key=>$subvalue){
if(!in_array($subvalue,$optionLabelArray)){
$setValue[]=$this->updateSourceAndReturnId($field,$subvalue);
}
}
}
/**This is the case of single select**/
else {
$setValue = null;
$newOptionLabelArray=array();
foreach ($options as $item) {
if ($item['label'] == $value) {
$setValue = $item['value'];
array_push($optionLabelArray,$item['label']); /**Adding Reference to worked attribute option**/
}
}
/**
* Checking in the current XML column value if all values were used in the above loop or not
* If not used then they are new options value, then new option is created and then assigned.
**/
if(!in_array($value,$optionLabelArray)){
$setValue=$this->updateSourceAndReturnId($field,$value);
}
}
}
}
/**CODE MODIFICATION ENDS HERE*/
$product->setData($field, $setValue);
}
if (!$product->getVisibility()) {
$product->setVisibility(Mage_Catalog_Model_Product_Visibility::VISIBILITY_NOT_VISIBLE);
}
$stockData = array();
$inventoryFields = isset($this->_inventoryFieldsProductTypes[$product->getTypeId()])
? $this->_inventoryFieldsProductTypes[$product->getTypeId()]
: array();
foreach ($inventoryFields as $field) {
if (isset($importData[$field])) {
if (in_array($field, $this->_toNumber)) {
$stockData[$field] = $this->getNumber($importData[$field]);
}
else {
$stockData[$field] = $importData[$field];
}
}
}
$product->setStockData($stockData);
$imageData = array();
foreach ($this->_imageFields as $field) {
if (!empty($importData[$field]) &amp;amp;&amp;amp; $importData[$field] != 'no_selection') {
if (!isset($imageData[$importData[$field]])) {
$imageData[$importData[$field]] = array();
}
$imageData[$importData[$field]][] = $field;
}
}
foreach ($imageData as $file => $fields) {
try {
$product->addImageToMediaGallery(Mage::getBaseDir('media') . DS . 'import' . $file, $fields);
}
catch (Exception $e) {}
}
$product->setIsMassupdate(true);
$product->setExcludeUrlRewrite(true);
$product->save();
return true;
}
/**
* Updates the source of the attribute by the current new value in the XML column, and returns
* the id of the newly created option.
*
* @param string $attribute_code
* @param string $newOption
* @return int|string New Option Id
*/
public function updateSourceAndReturnId($attribute_code,$newOption){
$attribute_model = Mage::getModel('eav/entity_attribute');
$attribute_options_model= Mage::getModel('eav/entity_attribute_source_table') ;
$attribute_code = $attribute_model->getIdByCode('catalog_product', $attribute_code);
$attribute = $attribute_model->load($attribute_code);
$attribute_table = $attribute_options_model->setAttribute($attribute);
try{
$value['option'] = array(trim($newOption),trim($newOption));
$result = array('value' => $value);
$attribute->setData('option',$result);
$attribute->save();
}
catch(Exception $e){}
$options = $attribute_options_model->getAllOptions(false);
foreach($options as $option)
{
if ($option['label'] == $newOption)
{
return $option['value'];
}
}
return "";
}
}
?>
Please note the /**CODE MODIFICATION ENDS HERE*/ block. Most of the description is on the code. I got it working, feel free to ask if you got any problem. I am also working on importing custom option. I’ll soon make a post in it as well.
Here’s the sample import sheet with request from the reader
I’m looking for a way to upload attribute option values in Magento. For some attributes I have like 4000/5000 different values, so I don’t want to enter those manually.
I stumbled upon your website. As I understand, the code above enables you to upload products and upload missing attribute options at the same time?
The part of creating an advanced profile I get, but what do I have to do with the second part “the model”?
And isn’t there an easy way to simply upload all the attribute values before I upload the product? Because I allready have csv with all attribute values…
As you can see in advanced profile there is a node called adapter, which is the model name for importing product. You can just create a new Model and simply use the above code in that model.
That should work, in profile, but if you want to first import attributes only then you can use the function updateSourceAndReturnId(), to do that but you need to read the CSV and send params accordingly. Hope this helps.
I implemented your code into an existing adapter I put together. This custom adapter adds the functionality to add categories on the fly with yours to add attribute option values. However I am running into a problem. It seems that when my comma delimited list of attribute values has spaces between the value and the delimiter all seems to work (ie. 2005 , 2006 , 2007). When I have a list however that looks like this “2005,2006,2007″ or like this “2005, 2006, 2007″ then the adapter adds the missing attribute option values as one new value. For example if currently my attribute has the option 2007. Then using the examples above 2005 and 2006 would be added as one new attribute option value looking like this “2005, 2006″. Again this doesn’t seem to occur when spaces exist on either side of the comma. Is this supposed to work this way? Am I potentially missing something? I can provide my code if your interested in taking a peak. Does this have anything to do with them be integer types? My attribute type is a multiselect by the way.
Thanks for the help.
Jeremy
Yeha I recently found out that too.. try using trim function before passing the value to all the call for updateSourceAndReturnId function.
It would then look like
In case of Multiselect
$this->updateSourceAndReturnId ( $field, trim ( $subvalue ) );
In case of Single Select.
$this->updateSourceAndReturnId ( $field, trim ( $value ) );
This should resolve the problem
Also after this condition
if($attribute->getFrontendInput () == ‘multiselect’){
// Replace this block with this.
}
Change the code block to this
$temp= split(self::MULTI_DELIMITER, $value);
$value=array();
foreach($temp as $values){
$value[]=trim($values);
}
$isArray = true;
$setValue = array ();
I’ve updated the source as well
Thanks for your code.
In the end of your Actions XML you have the line:
save
The standard product import profile has the line:
parse
Can you tell me what the effect of “save” or “parse” is?
Are you clear about the product importing general procedure..? if..not..plz first see this..i think it will be clear..after that..
http://subesh.com.np/2009/12/general-workflow-of-magentos-importexport-dataflow
Thanx for the link. I read it and understand the following order: “you need to first fetch the data from the source and process it and then save to its destination.”. But i dont know how to interpret the xml code (i’m a xml noob).
The original product import xml code in magento (see below) is the same as your xml code…only the last line is different. I still dont understand the consequences. Is this difference causing the ability of your xml to save attribute options? and/or are there other consequences?
true
1
catalog/convert_adapter_product
parse
(I see that i can’t show the code in my reply)
But the last line of code is the only difference:
your code: var name= “method” save /var
original code: var name = “method” parse /var
why is the original parsing, and your code saving, as a last step?
There may be two possibilities:
1. In the original code there must be another root node (action block) that defines adapter and may have save method called…but in this code the adapter is defined inside the parser action block and the save method is called of the adapter.
2. May be the implementation has changed in later Magento versions, I think i used this for 1.3.2.2..which version are you using?
Did the action XML not work for you? or are you just asking for the sake of understanding?
I’m using Magento version 1.3.2.4
I haven’t tested your code yet, but i was indeed just asking for the sake of understanding (i dont want to mess something up when i’m importing 60.000 products).
It will work i’ve tested over 6000 Products…working well…..
Well…i tried you script but i when i run the profile i get the message: “Method “save” not defined in adapter catalog/convert_adapter_import”.
Import.php i saved in: “app\code\local\Import\Catalog\Model\Convert\Adapter”
config.xml in “app\code\local\Import\Catalog\etc” and has the rewrite: “convert_adapter_import>Import_Catalog_Model_Convert_Adapter_Import
local
true
</config
I assume that hte method "save" is defined in your code, am i correct? Do you have an idea what i did wrong?
Well…i tried you script but i when i run the profile i get the message: “Method “save” not defined in adapter catalog/convert_adapter_import”.
Import.php i saved in: “app\code\local\Import\Catalog\Model\Convert\Adapter”
config.xml in “app\code\local\Import\Catalog\etc” and has the rewrite: “convert_adapter_import>Import_Catalog_Model_Convert_Adapter_Import<convert_adapter_import".
I also addes a module "Impor_All.xml" in "app\etc\modules"
I assume that hte method "save" is defined in your code, am i correct? Do you have an idea what i did wrong?
(unfortunately i cant post my code here)
Ok…i forgot to change the class in the import.php file….but now i get for every product i try to upload the message: Notice: “Undefined variable: newOptionLabelArray in /home/admin/domains/example.com/public_html/magento/app/code/local/Import/Catalog/Model/Convert/Adapter/Import.php on line 165″.
I used your code from the download source [updated] link.
Well just replace $newoptionLabelArray with $optionLabelArray in the code.
Only on line 165, or also on other lines? sorry….i’m not a prgrammer
when i only change it on line 165, and when i change all the $newoptionLabelArray to $OptionLabelArray, i get the message: Notice: Undefined variable: OptionLabelArray in /home/admin/domains/kranenshop.nl/public_html/magento/app/code/local/Import/Catalog/Model/Convert/Adapter/Import.php on line 165.
Hi there I have been trying to use this with Magento ver. 1.3.2.4.
I have it set up… only problem is that the model path.. I just can’t get it right Ihave put the model in the same directory as the xml file “var/import”
I have set the “var/import/import.php”
I have also experimented with just putting it in the magento root directory.
(I have tried using a path like this
“/home/myserver/public_html/magento/var/import/import.php”
Which is the same as the path where the XML file IS found, No problem.. But it does not work for the import.php)
I am sorry to be to be dense … would you please tell me litterally what the correct value I should put in the path, to find the import.php in var/import .. or any other place. The file is deffinately there.
Please help.. thank you very much in advance, for all your kind efforts.
Okay… well I fouind a beter place to put the model
“magento/app/code/core/Mage/Catalog/Model/Convert/Adapter/Import.php”
kept the file name the same, and the content the same as supplied
and I put this in the path of the XML “catalog/convert_adapter_product”
I tried many different possibilities, and t hat is the only one that work at all. How ever I think that it maybe running a different script.
Can anyone please please advise me, I have got as far as I can with this.
Sorry to be a dullard, just one last nudge would be much appreciated.
Thanks
Hi, Sorry for the late response ..
The “best” place to keep your Model file is your Module’s Model folder. Have you created a custom module in Magento before? If not then plz see this
http://www.magentocommerce.com/wiki/custom_module_with_custom_database_table
Use the Module creator found there and run to create a basic Custom module extension. Create Import.php inside the Model folder of this module and in your Profile XML will have the Model path to this file.
Suppose you have Create a Namespace_Module as Mynamespace_Mymodule then the model path will be like mymodule/import.
This might help you I guess..if not plz feel free to ask again!
Hi Subesh,
Thanks for all your hard work.
Can you provide either a sample CSV file or tell me what Headers we would add for the attribute options?
Thanks,
Ted
Also will this work for Magento ver 1.4.01?
Thanks again
Waiting for feedbacks
Hi Subesh Thanks. I will be doing the testing in ver1.4.01.. What really could help is if someone could:
1. provide a sample CSV file
2. What Headings we would add for the spreadsheet especially for attribute options
Thanks in advance for all your help, and thank you Subesh for your knowledge sharing on this blog.
I’ve uploaded the XLS sheet here plz check
http://spreadsheets.google.com/ccc?key=0Aji4wlRfG_gjdFEzRF9nVWFvZTRlMEZ5WGcxUWV4ZGc&hl=en
I am testing with Magento 1.4.0.1 and I am getting the following error after import has started for each product:
* Notice: Array to string conversion in /home/administrator/Projects/magento/htdocs/app/code/local/SURE/ImportA/Model/Import.php on line 216
then no records were imported…
* Processed 100% 7/7 records
* Imported 0 records
* Finished profile execution.
The format of the CSV is what Magento 1.4.0.1 has exported, I just added a couple of lines for testing so I guess the format of the CSV is correct.
Your help would be very much appreciated… thanks!
I changed line 216 of import.php from
$setValue=$this->updateSourceAndReturnId($field,trim($value));
to
$setValue=$this->updateSourceAndReturnId($field,$value);
and now it seems to be importing ok… Could you confirm if this was the right thing to do and if you can see any potential problems?
Thanks again for a great bit a work… will save me loads of time.
I just had 2 days of clueless guessing why I keep getting:
Method “save” not defined in adapter
So for future reference you should know that the adapter path is always lowercase and is built like that:
_ means “/”
/ means “/Model”
PHP file should be lowercase and without “.php” extension
So when you put the path to the Import.php inside your Model directory, if the directory looks like that:
/app/code/local/Namespace/Module/Model/Import.php
the adapter path would be:
module/import
I also recommend using the Module Creator from here:
http://www.magentocommerce.com/magento-connect/Daniel+Nitz/extension/1108/modulecreator
Other than that, Thank you Subesh! you saved tons of manual work.
Help! just as I thought I figured it out I found that all the options are just “Array” so now I got thousands of “array” options but not a single proper one.
Yigal, did you figure out how to get a proper single select?
No, I gave up on this method and used this one:
http://www.magentocommerce.com/boards/viewthread/9391/P0/
Do the attributes have to already exist and be assigned to the correct attribute set for this to work? I got this working and it imports products but there are still many attributes missing after the import.