Subesh Pokhrel's Blog – Magento Development Tips,PHP,Google Maps
PHP & Magento Tips & Tutorials
Share

As you may know or not ! Magento is the fastest growing eCommerce plateform, nowonder because Magento has a lots of flexibility in it. You can feel that too if you are a just a user and as for me as Programmer I just Love it. One of the salient feature that makes Magento a leader in eCommerce software is its ability to Import and Export data to an from the system.

Magento gives you a default interface to import or export data, but you will need the specific format of your data.It is easy to customize that as well, but today lets be interested in standard import, but with one small (Yet important) modification. The important modification in the standard code is this code will be able to options to the attribute of Select/Multiselect/Radio/Checkbox input type, while the product is being imported.

By default Magento only imports the values of the option, of any attribute, which are already avaliable. But this code can be used to create option at run time if it does not exits and assign to the product, during import. For doing this we need to first set up an Advanced Profile of Export/Import. Go to System–>Import/Export–>Advanced Profile and create new Profile. Then in the Action XML paste this.


<!--
Path=Path of the file relative to ROOT
filename= File name of the XML(MS EXCEL 2003) where import data is saved in Magento's Format
format	= XML
You can use CSV as well.
-->
<action type="dataflow/convert_adapter_io" method="load">
    <var name="type">file</var>
    <var name="path">var/import</var>
    <var name="filename"><![CDATA[products.xml]]></var>
    <var name="format"><![CDATA[xml]]></var>
</action>

<!--
adapter= Your Model path
-->
<action type="dataflow/convert_parser_xml_excel" method="parse">
    <var name="single_sheet"><![CDATA[]]></var>
    <var name="fieldnames">true</var>
    <var name="store"><![CDATA[0]]></var>
    <var name="number_of_records">1</var>
    <var name="decimal_separator"><![CDATA[.]]></var>
    <var name="adapter">yourmodel/import</var>
    <var name="method">save</var>
</action>

The Model defined in the adapter should be like this. You can create a new module for this.


<?php
class Namespace_Module_Model_Import extends Mage_Catalog_Model_Convert_Adapter_Product {

	/**
	 * Save product (import)
	 * @param array $importData
	 * @throws Mage_Core_Exception
	 * @return bool
	 */
	public function saveRow(array $importData)
	{
		$product = $this->getProductModel()
		->reset();

		if (empty($importData['store'])) {
			if (!is_null($this->getBatchParams('store'))) {
				$store = $this->getStoreById($this->getBatchParams('store'));
			} else {
				$message = Mage::helper('catalog')->__('Skip import row, required field "%s" not defined', 'store');
				Mage::throwException($message);
			}
		}
		else {
			$store = $this->getStoreByCode($importData['store']);
		}

		if ($store === false) {
			$message = Mage::helper('catalog')->__('Skip import row, store "%s" field not exists', $importData['store']);
			Mage::throwException($message);
		}

		if (empty($importData['sku'])) {
			$message = Mage::helper('catalog')->__('Skip import row, required field "%s" not defined', 'sku');
			Mage::throwException($message);
		}
		$product->setStoreId($store->getId());
		$productId = $product->getIdBySku($importData['sku']);

		if ($productId) {
			$product->load($productId);
		}
		else {
			$productTypes = $this->getProductTypes();
			$productAttributeSets = $this->getProductAttributeSets();

			/**
			 * Check product define type
			 */
			if (empty($importData['type']) || !isset($productTypes[strtolower($importData['type'])])) {
				$value = isset($importData['type']) ? $importData['type'] : '';
				$message = Mage::helper('catalog')->__('Skip import row, is not valid value "%s" for field "%s"', $value, 'type');
				Mage::throwException($message);
			}
			$product->setTypeId($productTypes[strtolower($importData['type'])]);
			/**
			 * Check product define attribute set
			 */
			if (empty($importData['attribute_set']) || !isset($productAttributeSets[$importData['attribute_set']])) {
				$value = isset($importData['attribute_set']) ? $importData['attribute_set'] : '';
				$message = Mage::helper('catalog')->__('Skip import row, is not valid value "%s" for field "%s"', $value, 'attribute_set');
				Mage::throwException($message);
			}
			$product->setAttributeSetId($productAttributeSets[$importData['attribute_set']]);

			foreach ($this->_requiredFields as $field) {
				$attribute = $this->getAttribute($field);
				if (!isset($importData[$field]) &amp;amp;amp;&amp;amp;amp; $attribute &amp;amp;amp;&amp;amp;amp; $attribute->getIsRequired()) {
					$message = Mage::helper('catalog')->__('Skip import row, required field "%s" for new products not defined', $field);
					Mage::throwException($message);
				}
			}
		}

		$this->setProductTypeInstance($product);

		if (isset($importData['category_ids'])) {
			$product->setCategoryIds($importData['category_ids']);
		}

		foreach ($this->_ignoreFields as $field) {
			if (isset($importData[$field])) {
				unset($importData[$field]);
			}
		}

		if ($store->getId() != 0) {
			$websiteIds = $product->getWebsiteIds();
			if (!is_array($websiteIds)) {
				$websiteIds = array();
			}
			if (!in_array($store->getWebsiteId(), $websiteIds)) {
				$websiteIds[] = $store->getWebsiteId();
			}
			$product->setWebsiteIds($websiteIds);
		}

		if (isset($importData['websites'])) {
			$websiteIds = $product->getWebsiteIds();
			if (!is_array($websiteIds)) {
				$websiteIds = array();
			}
			$websiteCodes = split(',', $importData['websites']);
			foreach ($websiteCodes as $websiteCode) {
				try {
					$website = Mage::app()->getWebsite(trim($websiteCode));
					if (!in_array($website->getId(), $websiteIds)) {
						$websiteIds[] = $website->getId();
					}
				}
				catch (Exception $e) {}
			}
			$product->setWebsiteIds($websiteIds);
			unset($websiteIds);
		}

		foreach ($importData as $field => $value) {
			if (in_array($field, $this->_inventoryFields)) {
				continue;
			}
			if (in_array($field, $this->_imageFields)) {
				continue;
			}

			$attribute = $this->getAttribute($field);

			if (!$attribute) {
				continue;
			}

			$isArray = false;
			$setValue = $value;

			if ($attribute->getFrontendInput() == 'multiselect') {
				$value = split(self::MULTI_DELIMITER, $value);
				$isArray = true;
				$setValue = array();
			}

			if ($value &amp;amp;amp;&amp;amp;amp; $attribute->getBackendType() == 'decimal') {
				$setValue = $this->getNumber($value);
			}

			/**CODE MODIFICATION STARTS HERE*/

			$optionLabelArray=array();
			if ($attribute->usesSource()) {
			 $options = $attribute->getSource()->getAllOptions(false);

			 /**
			  * Update the Source of the attribute when source has no options.
			  */
			 if(count($options)<1){
			 	if($isArray){
			 		foreach($value as $key=>$subvalue){
			 			if(!in_array($subvalue,$newOptionLabelArray)){
			 				$setValue[]=$this->updateSourceAndReturnId($field,$subvalue);
			 				array_push($newOptionLabelArray,$subvalue);
			 			}
			 		}
			 	}
			 	else{
			 		if(!in_array($value,$newOptionLabelArray)){
			 			$setValue=$this->updateSourceAndReturnId($field,$value);
			 			array_push($newOptionLabelArray,$value);
			 		}
			 	}
			 }

			 /**
			  * Work on the source when it has options
			  */
			 else{
			 	/**
			 	 * This is the case of Multi-Select
			 	 */
			 	if ($isArray) {
			 		foreach ($options as $item) {
			 			/** Setting the option's ID if Label matches with the current value of XML column.*/
			 			if (in_array($item['label'], $value)) {
			 				$setValue[] = trim($item['value']);
			 				array_push($optionLabelArray,$item['label']); /**Adding Reference to worked attribute option**/
			 			}
			 		}

			 		/**
			 		 *  Checking in the current XML column value if all values were used in the above loop or not
			 		 *  If not used then they are new options value, then new option is created and then assigned.
			 		 **/

			 		foreach($value as $key=>$subvalue){
			 			if(!in_array($subvalue,$optionLabelArray)){
			 				$setValue[]=$this->updateSourceAndReturnId($field,$subvalue);
			 			}
			 		}

			 	}
			 	/**This is the case of single select**/

			 	else {
			 		$setValue = null;
			 		$newOptionLabelArray=array();
			 		foreach ($options as $item) {
			 			if ($item['label'] == $value) {
			 				$setValue = $item['value'];
			 				array_push($optionLabelArray,$item['label']); /**Adding Reference to worked attribute option**/
			 			}
			 		}
			 		/**
			 		 *  Checking in the current XML column value if all values were used in the above loop or not
			 		 *  If not used then they are new options value, then new option is created and then assigned.
			 		 **/
			 		if(!in_array($value,$optionLabelArray)){
			 			$setValue=$this->updateSourceAndReturnId($field,$value);
			 		}
			 	}
			 }
			}

			/**CODE MODIFICATION ENDS HERE*/

			$product->setData($field, $setValue);
		}

		if (!$product->getVisibility()) {
			$product->setVisibility(Mage_Catalog_Model_Product_Visibility::VISIBILITY_NOT_VISIBLE);
		}

		$stockData = array();
		$inventoryFields = isset($this->_inventoryFieldsProductTypes[$product->getTypeId()])
		? $this->_inventoryFieldsProductTypes[$product->getTypeId()]
		: array();
		foreach ($inventoryFields as $field) {
			if (isset($importData[$field])) {
				if (in_array($field, $this->_toNumber)) {
					$stockData[$field] = $this->getNumber($importData[$field]);
				}
				else {
					$stockData[$field] = $importData[$field];
				}
			}
		}
		$product->setStockData($stockData);

		$imageData = array();
		foreach ($this->_imageFields as $field) {
			if (!empty($importData[$field]) &amp;amp;amp;&amp;amp;amp; $importData[$field] != 'no_selection') {
				if (!isset($imageData[$importData[$field]])) {
					$imageData[$importData[$field]] = array();
				}
				$imageData[$importData[$field]][] = $field;
			}
		}

		foreach ($imageData as $file => $fields) {
			try {
				$product->addImageToMediaGallery(Mage::getBaseDir('media') . DS . 'import' . $file, $fields);
			}
			catch (Exception $e) {}
		}

		$product->setIsMassupdate(true);
		$product->setExcludeUrlRewrite(true);
		$product->save();

		return true;
	}

	/**
	 * Updates the source of the attribute by the current new value in the XML column, and returns
	 * the id of the newly created option.
	 *
	 * @param string $attribute_code
	 * @param string $newOption
	 * @return int|string New Option Id
	 */
	public function updateSourceAndReturnId($attribute_code,$newOption){
		$attribute_model        = Mage::getModel('eav/entity_attribute');
		$attribute_options_model= Mage::getModel('eav/entity_attribute_source_table') ;

		$attribute_code         = $attribute_model->getIdByCode('catalog_product', $attribute_code);
		$attribute              = $attribute_model->load($attribute_code);
		$attribute_table        = $attribute_options_model->setAttribute($attribute);
		try{
			$value['option'] = array(trim($newOption),trim($newOption));
			$result = array('value' => $value);
			$attribute->setData('option',$result);
			$attribute->save();
		}
		catch(Exception $e){}
		$options = $attribute_options_model->getAllOptions(false);
		foreach($options as $option)
		{
			if ($option['label'] == $newOption)
			{
				return $option['value'];
			}
		}
		return "";
	}

}

?>

Please note the /**CODE MODIFICATION ENDS HERE*/ block. Most of the description is on the code. I got it working, feel free to ask if you got any problem. I am also working on importing custom option. I’ll soon make a post in it as well.

Download Source[Updated]

Here’s the sample import sheet with request from the reader

Sample Sheet

  • Share/Bookmark

You may also be interested

Comments

26 Responses to “Product Import In Magento Alongwith Updating The Attribute’s Options”

  1. I’m looking for a way to upload attribute option values in Magento. For some attributes I have like 4000/5000 different values, so I don’t want to enter those manually.

    I stumbled upon your website. As I understand, the code above enables you to upload products and upload missing attribute options at the same time?

    The part of creating an advanced profile I get, but what do I have to do with the second part “the model”?

    And isn’t there an easy way to simply upload all the attribute values before I upload the product? Because I allready have csv with all attribute values…

    • Subesh Pokhrel says:

      As you can see in advanced profile there is a node called adapter, which is the model name for importing product. You can just create a new Model and simply use the above code in that model.

      That should work, in profile, but if you want to first import attributes only then you can use the function updateSourceAndReturnId(), to do that but you need to read the CSV and send params accordingly. Hope this helps.

  2. Jeremy Moore says:

    I implemented your code into an existing adapter I put together. This custom adapter adds the functionality to add categories on the fly with yours to add attribute option values. However I am running into a problem. It seems that when my comma delimited list of attribute values has spaces between the value and the delimiter all seems to work (ie. 2005 , 2006 , 2007). When I have a list however that looks like this “2005,2006,2007″ or like this “2005, 2006, 2007″ then the adapter adds the missing attribute option values as one new value. For example if currently my attribute has the option 2007. Then using the examples above 2005 and 2006 would be added as one new attribute option value looking like this “2005, 2006″. Again this doesn’t seem to occur when spaces exist on either side of the comma. Is this supposed to work this way? Am I potentially missing something? I can provide my code if your interested in taking a peak. Does this have anything to do with them be integer types? My attribute type is a multiselect by the way.

    Thanks for the help.
    Jeremy

    • Subesh Pokhrel says:

      Yeha I recently found out that too.. try using trim function before passing the value to all the call for updateSourceAndReturnId function.
      It would then look like

      In case of Multiselect
      $this->updateSourceAndReturnId ( $field, trim ( $subvalue ) );

      In case of Single Select.
      $this->updateSourceAndReturnId ( $field, trim ( $value ) );

      This should resolve the problem

      Also after this condition
      if($attribute->getFrontendInput () == ‘multiselect’){
      // Replace this block with this.
      }

      Change the code block to this

      $temp= split(self::MULTI_DELIMITER, $value);
      $value=array();
      foreach($temp as $values){
      $value[]=trim($values);
      }
      $isArray = true;
      $setValue = array ();

    • Subesh Pokhrel says:

      I’ve updated the source as well

  3. Thanks for your code.
    In the end of your Actions XML you have the line:
    save
    The standard product import profile has the line:
    parse
    Can you tell me what the effect of “save” or “parse” is?

    • Subesh Pokhrel says:

      Are you clear about the product importing general procedure..? if..not..plz first see this..i think it will be clear..after that..

      http://subesh.com.np/2009/12/general-workflow-of-magentos-importexport-dataflow

      • Thanx for the link. I read it and understand the following order: “you need to first fetch the data from the source and process it and then save to its destination.”. But i dont know how to interpret the xml code (i’m a xml noob).

        The original product import xml code in magento (see below) is the same as your xml code…only the last line is different. I still dont understand the consequences. Is this difference causing the ability of your xml to save attribute options? and/or are there other consequences?

        true

        1

        catalog/convert_adapter_product
        parse

  4. (I see that i can’t show the code in my reply)
    But the last line of code is the only difference:
    your code: var name= “method” save /var
    original code: var name = “method” parse /var
    why is the original parsing, and your code saving, as a last step?

    • Subesh Pokhrel says:

      There may be two possibilities:

      1. In the original code there must be another root node (action block) that defines adapter and may have save method called…but in this code the adapter is defined inside the parser action block and the save method is called of the adapter.
      2. May be the implementation has changed in later Magento versions, I think i used this for 1.3.2.2..which version are you using?

      Did the action XML not work for you? or are you just asking for the sake of understanding?

      • I’m using Magento version 1.3.2.4
        I haven’t tested your code yet, but i was indeed just asking for the sake of understanding (i dont want to mess something up when i’m importing 60.000 products).

        • Subesh Pokhrel says:

          It will work i’ve tested over 6000 Products…working well…..

          • Well…i tried you script but i when i run the profile i get the message: “Method “save” not defined in adapter catalog/convert_adapter_import”.
            Import.php i saved in: “app\code\local\Import\Catalog\Model\Convert\Adapter”
            config.xml in “app\code\local\Import\Catalog\etc” and has the rewrite: “convert_adapter_import>Import_Catalog_Model_Convert_Adapter_Import

            local
            true

            </config
            I assume that hte method "save" is defined in your code, am i correct? Do you have an idea what i did wrong?

  5. Well…i tried you script but i when i run the profile i get the message: “Method “save” not defined in adapter catalog/convert_adapter_import”.

    Import.php i saved in: “app\code\local\Import\Catalog\Model\Convert\Adapter”

    config.xml in “app\code\local\Import\Catalog\etc” and has the rewrite: “convert_adapter_import>Import_Catalog_Model_Convert_Adapter_Import<convert_adapter_import".

    I also addes a module "Impor_All.xml" in "app\etc\modules"

    I assume that hte method "save" is defined in your code, am i correct? Do you have an idea what i did wrong?

    (unfortunately i cant post my code here)

  6. Ok…i forgot to change the class in the import.php file….but now i get for every product i try to upload the message: Notice: “Undefined variable: newOptionLabelArray in /home/admin/domains/example.com/public_html/magento/app/code/local/Import/Catalog/Model/Convert/Adapter/Import.php on line 165″.

    I used your code from the download source [updated] link.

  7. Subesh Pokhrel says:

    Well just replace $newoptionLabelArray with $optionLabelArray in the code.

    • Only on line 165, or also on other lines? sorry….i’m not a prgrammer :)

    • when i only change it on line 165, and when i change all the $newoptionLabelArray to $OptionLabelArray, i get the message: Notice: Undefined variable: OptionLabelArray in /home/admin/domains/kranenshop.nl/public_html/magento/app/code/local/Import/Catalog/Model/Convert/Adapter/Import.php on line 165.

  8. Hi there I have been trying to use this with Magento ver. 1.3.2.4.

    I have it set up… only problem is that the model path.. I just can’t get it right Ihave put the model in the same directory as the xml file “var/import”

    I have set the “var/import/import.php”

    I have also experimented with just putting it in the magento root directory.

    (I have tried using a path like this

    “/home/myserver/public_html/magento/var/import/import.php”

    Which is the same as the path where the XML file IS found, No problem.. But it does not work for the import.php)

    I am sorry to be to be dense … would you please tell me litterally what the correct value I should put in the path, to find the import.php in var/import .. or any other place. The file is deffinately there.

    Please help.. thank you very much in advance, for all your kind efforts.

  9. Okay… well I fouind a beter place to put the model

    “magento/app/code/core/Mage/Catalog/Model/Convert/Adapter/Import.php”

    kept the file name the same, and the content the same as supplied

    and I put this in the path of the XML “catalog/convert_adapter_product”

    I tried many different possibilities, and t hat is the only one that work at all. How ever I think that it maybe running a different script.

    Can anyone please please advise me, I have got as far as I can with this.

    Sorry to be a dullard, just one last nudge would be much appreciated.

    Thanks

    • Subesh Pokhrel says:

      Hi, Sorry for the late response ..

      The “best” place to keep your Model file is your Module’s Model folder. Have you created a custom module in Magento before? If not then plz see this

      http://www.magentocommerce.com/wiki/custom_module_with_custom_database_table

      Use the Module creator found there and run to create a basic Custom module extension. Create Import.php inside the Model folder of this module and in your Profile XML will have the Model path to this file.

      Suppose you have Create a Namespace_Module as Mynamespace_Mymodule then the model path will be like mymodule/import.

      This might help you I guess..if not plz feel free to ask again!

  10. Hi Subesh,
    Thanks for all your hard work.
    Can you provide either a sample CSV file or tell me what Headers we would add for the attribute options?
    Thanks,
    Ted

  11. Also will this work for Magento ver 1.4.01?
    Thanks again

  12. Hi Subesh Thanks. I will be doing the testing in ver1.4.01.. What really could help is if someone could:
    1. provide a sample CSV file
    2. What Headings we would add for the spreadsheet especially for attribute options

    Thanks in advance for all your help, and thank you Subesh for your knowledge sharing on this blog.

Write a Comment

Search engine optimization by SEO Design Solutions