/**Using the Previously created DB**/
mysql> use magento

/**Import the Dumpped SQL**/
mysql>source pathToMagentoRoot/mySql.sql

So after the command is executed I had the whole DB on production enviornment. Similarly we can also import dumpped data of Production on Local database running on WAMP server.
To import Dumpped SQL data on WAMP, first create a database, say magento and then use the following commands

/**
	Using the Previously created DB.
	-p password part is optional if you have not configured password for MYSQL locally
	-u Username (root by default)
**/

mysql> C:\wamp\bin\mysql\mysql5.0.51b\bin>mysql.exe -use magento -u root -p password

/**Import the Dumpped SQL**/
mysql>source pathToSQLSource/mySql.sql

By Now we have the database synced between Local and Production. Next is configuring the database name used by Magento. Browse to app/etc/local.xml and change in the following part.

    <connection>
                    <host><![CDATA[localhost]]></host>
                    <username><![CDATA[username_here]]></username>
                    <password><![CDATA[password_here]]></password>
                    <dbname><![CDATA[databasename_here]]></dbname>
                    <active>1</active>
    </connection>

Finally, browsed through table core_config_data and find value of path: web/unsecure/base_url and path: web/secure/base_url and change their values accordingly.

Now I had my Shop enviorment switched. Most work for you as well. LOL.

General Work Flow (Profile)

During Import/Export there are normally two “Actors” involved. Actors may be the data source (Actor: Source) and the final destination of the data (Actor:Destination). While you are importing data into your Magento Sytem, the Source is the CSV or XML file while exporting it is the other way round. In any case (Import or Export) you need to first fetch the data from the source and process it and then save to its destination.

So Basically there are three stages in product import. But the processing part has been split into two. Processing part has been divided into two processes namely Mapper and Parser. I’ll come to the detailed description later. So along with the extended processes, lets call there are four “Actions” occuring during Import or Export.For every “Action” to run there are seperate individual Models. And the defination of all these individual Models for each Action during the Import/Export Process (Dataflow) is called “Profile“.

Here’s the basic workflow for Dataflow.

General Workflow

That was the laymen description of Dataflow, lets see what are the Jargons used in Magento to call all the Models used in Dataflow. The Models used for all actions are listed below according to how the Dataflow works.

1. Adapter
2. Parser
3. Mapper
4. Adapter (Yes twice)
5. Validator (Not Implemented yet in Magento, left for future implemention)

Adapter

To understand what is Adapter, lets see Magento as a System and all the resources it uses during Import/Export as external source. So external source in Most of the cases can be either File or your Database (DB). Other external source include Webservice. I’ven’t came across other yet. So what Adapter does is it provides the channel through which the data actually flows in/out of the Magento System. As you can see in the images below is that there is always two Adapter involved. One that reads data from the file external resource and another that saves the data into the DB external resource. Now you should be clear why I wrote Adapter twice. If you are still wondering..then you can comment below.

Case: Export

Case: Import

Magento has provides us some Default Adapter to read/write from/to external resources. Here are the Adapters listed.

Adapter For Files : Mage_Dataflow_Model_Convert_Adapter_Io

Adapter For Customer(DB) : Mage_Customer_Model_Convert_Adapter_Customer

Adapter For Product (DB) : Mage_Catalog_Model_Convert_Adapter_Product

Since the work of Adapter is to just Read or Write it has basically two Methods load() and save().

Parser

What parser does is that it just converts the raw data (Human Readable) from the source into Magento’s Format (Arrays and Setting Values of Magento’s Model) or from Magento’s Format into Human readable Format. Just to understand you can think of the CSV files seperated by comma changed into Arrays. After conversion is over it then saves the data to the Batch.Batch, what is Batch? if this is what you are thinking, here is your answer. Every row in CSV,XML or Row of the DB during the dataflow are saved as different rows in a table dataflow_batch_import or dataflow_batch_export. So if there are 100 rows in your CSV file you are importing into Magento, the batch will have 100 rows of parsed data in the table dataflow_batch_import. During the final stage of the Dataflow’s run this data is referred.

Magento provides some default Parses as well.

Parser for CSV files : Mage_Dataflow_Model_Convert_Parser_Csv

Parser for XML files : Mage_Dataflow_Model_Convert_Parser_Xml_Excel

Parser for Product : Mage_Catalog_Model_Convert_Parser_Product

Parser for Customer : Mage_Customer_Model_Convert_Parser_Customer

In all these classes there are two main Methods which deservers to be stated here. parse() and unparse(). If you are Importing then parse() should be defined in your profile, because you are parsing the data that is required by the System. In case of Exporting, you already have the parsed data of the system as source, you will then need to unparse() the data into Human readable format. You can refer to images above to understand more clearly.

So by the end of this during the profile run you will have Parsed/Unparsed data saved in the batch table, in you hand.

Mapper

Magento’s Dataflow is so flexible that you can set the header of the CSV or XML files that can mapped to the Magento’s default attributes. Let’s take an example if you want to export the product’s name into CSV with the CSV header as “Product Name”, you can specify that mapping in your profile where the Mapper will act. And in this the Magento’s attribute code “name” will be mapped to your specified “Product Name”. To do all this mapping rules, Mapper is there.

It retrives the Parsed/Unparsed Data from the batch table, we got at the end of Parser’s work, Maps to the rule we defined and then again saves back to the same table. Easy!

There is one default Mapper for all

Mapper : Mage_Dataflow_Model_Convert_Mapper_Column

With one obvious method map().

At the end of this stage you will have your Parsed/Unparsed – Mapped Data in your dataflow table.

Adapter (Again)

At last the Adapter will use the dataflow’s batch data to save(). And after all the rows of the batch table for one batch is acted upon the profile is complete.

I guess this is the general way of explaining it, please comment if I am mistake, coz I am also new to this! LOL.

Magento gives you a default interface to import or export data, but you will need the specific format of your data.It is easy to customize that as well, but today lets be interested in standard import, but with one small (Yet important) modification. The important modification in the standard code is this code will be able to options to the attribute of Select/Multiselect/Radio/Checkbox input type, while the product is being imported.

By default Magento only imports the values of the option, of any attribute, which are already avaliable. But this code can be used to create option at run time if it does not exits and assign to the product, during import. For doing this we need to first set up an Advanced Profile of Export/Import. Go to System–>Import/Export–>Advanced Profile and create new Profile. Then in the Action XML paste this.

<!--
Path=Path of the file relative to ROOT
filename= File name of the XML(MS EXCEL 2003) where import data is saved in Magento's Format
format	= XML
You can use CSV as well.
-->
<action type="dataflow/convert_adapter_io" method="load">
    <var name="type">file</var>
    <var name="path">var/import</var>
    <var name="filename"><![CDATA[products.xml]]></var>
    <var name="format"><![CDATA1]></var>
</action>

<!--
adapter= Your Model path
-->
<action type="dataflow/convert_parser_xml_excel" method="parse">
    <var name="single_sheet"><![CDATA[]]></var>
    <var name="fieldnames">true</var>
    <var name="store"><![CDATA[0]]></var>
    <var name="number_of_records">1</var>
    <var name="decimal_separator"><![CDATA[.]]></var>
    <var name="adapter">yourmodel/import</var>
    <var name="method">save</var>
</action>

The Model defined in the adapter should be like this. You can create a new module for this.

<?php
class Namespace_Module_Model_Import extends Mage_Catalog_Model_Convert_Adapter_Product {

	/**
	 * Save product (import)
	 * @param array $importData
	 * @throws Mage_Core_Exception
	 * @return bool
	 */
	public function saveRow(array $importData)
	{
		$product = $this->getProductModel()
		->reset();

		if (empty($importData['store'])) {
			if (!is_null($this->getBatchParams('store'))) {
				$store = $this->getStoreById($this->getBatchParams('store'));
			} else {
				$message = Mage::helper('catalog')->__('Skip import row, required field "%s" not defined', 'store');
				Mage::throwException($message);
			}
		}
		else {
			$store = $this->getStoreByCode($importData['store']);
		}

		if ($store === false) {
			$message = Mage::helper('catalog')->__('Skip import row, store "%s" field not exists', $importData['store']);
			Mage::throwException($message);
		}

		if (empty($importData['sku'])) {
			$message = Mage::helper('catalog')->__('Skip import row, required field "%s" not defined', 'sku');
			Mage::throwException($message);
		}
		$product->setStoreId($store->getId());
		$productId = $product->getIdBySku($importData['sku']);

		if ($productId) {
			$product->load($productId);
		}
		else {
			$productTypes = $this->getProductTypes();
			$productAttributeSets = $this->getProductAttributeSets();

			/**
			 * Check product define type
			 */
			if (empty($importData['type']) || !isset($productTypes[strtolower($importData['type'])])) {
				$value = isset($importData['type']) ? $importData['type'] : '';
				$message = Mage::helper('catalog')->__('Skip import row, is not valid value "%s" for field "%s"', $value, 'type');
				Mage::throwException($message);
			}
			$product->setTypeId($productTypes[strtolower($importData['type'])]);
			/**
			 * Check product define attribute set
			 */
			if (empty($importData['attribute_set']) || !isset($productAttributeSets[$importData['attribute_set']])) {
				$value = isset($importData['attribute_set']) ? $importData['attribute_set'] : '';
				$message = Mage::helper('catalog')->__('Skip import row, is not valid value "%s" for field "%s"', $value, 'attribute_set');
				Mage::throwException($message);
			}
			$product->setAttributeSetId($productAttributeSets[$importData['attribute_set']]);

			foreach ($this->_requiredFields as $field) {
				$attribute = $this->getAttribute($field);
				if (!isset($importData[$field]) &amp;amp;amp;&amp;amp;amp; $attribute &amp;amp;amp;&amp;amp;amp; $attribute->getIsRequired()) {
					$message = Mage::helper('catalog')->__('Skip import row, required field "%s" for new products not defined', $field);
					Mage::throwException($message);
				}
			}
		}

		$this->setProductTypeInstance($product);

		if (isset($importData['category_ids'])) {
			$product->setCategoryIds($importData['category_ids']);
		}

		foreach ($this->_ignoreFields as $field) {
			if (isset($importData[$field])) {
				unset($importData[$field]);
			}
		}

		if ($store->getId() != 0) {
			$websiteIds = $product->getWebsiteIds();
			if (!is_array($websiteIds)) {
				$websiteIds = array();
			}
			if (!in_array($store->getWebsiteId(), $websiteIds)) {
				$websiteIds[] = $store->getWebsiteId();
			}
			$product->setWebsiteIds($websiteIds);
		}

		if (isset($importData['websites'])) {
			$websiteIds = $product->getWebsiteIds();
			if (!is_array($websiteIds)) {
				$websiteIds = array();
			}
			$websiteCodes = split(',', $importData['websites']);
			foreach ($websiteCodes as $websiteCode) {
				try {
					$website = Mage::app()->getWebsite(trim($websiteCode));
					if (!in_array($website->getId(), $websiteIds)) {
						$websiteIds[] = $website->getId();
					}
				}
				catch (Exception $e) {}
			}
			$product->setWebsiteIds($websiteIds);
			unset($websiteIds);
		}

		foreach ($importData as $field => $value) {
			if (in_array($field, $this->_inventoryFields)) {
				continue;
			}
			if (in_array($field, $this->_imageFields)) {
				continue;
			}

			$attribute = $this->getAttribute($field);

			if (!$attribute) {
				continue;
			}

			$isArray = false;
			$setValue = $value;

			if ($attribute->getFrontendInput() == 'multiselect') {
				$value = split(self::MULTI_DELIMITER, $value);
				$isArray = true;
				$setValue = array();
			}

			if ($value &amp;amp;amp;&amp;amp;amp; $attribute->getBackendType() == 'decimal') {
				$setValue = $this->getNumber($value);
			}

			/**CODE MODIFICATION STARTS HERE*/

			$optionLabelArray=array();
			if ($attribute->usesSource()) {
			 $options = $attribute->getSource()->getAllOptions(false);

			 /**
			  * Update the Source of the attribute when source has no options.
			  */
			 if(count($options)<1){
			 	if($isArray){
			 		foreach($value as $key=>$subvalue){
			 			if(!in_array($subvalue,$newOptionLabelArray)){
			 				$setValue[]=$this->updateSourceAndReturnId($field,$subvalue);
			 				array_push($newOptionLabelArray,$subvalue);
			 			}
			 		}
			 	}
			 	else{
			 		if(!in_array($value,$newOptionLabelArray)){
			 			$setValue=$this->updateSourceAndReturnId($field,$value);
			 			array_push($newOptionLabelArray,$value);
			 		}
			 	}
			 }

			 /**
			  * Work on the source when it has options
			  */
			 else{
			 	/**
			 	 * This is the case of Multi-Select
			 	 */
			 	if ($isArray) {
			 		foreach ($options as $item) {
			 			/** Setting the option's ID if Label matches with the current value of XML column.*/
			 			if (in_array($item['label'], $value)) {
			 				$setValue[] = trim($item['value']);
			 				array_push($optionLabelArray,$item['label']); /**Adding Reference to worked attribute option**/
			 			}
			 		}

			 		/**
			 		 *  Checking in the current XML column value if all values were used in the above loop or not
			 		 *  If not used then they are new options value, then new option is created and then assigned.
			 		 **/

			 		foreach($value as $key=>$subvalue){
			 			if(!in_array($subvalue,$optionLabelArray)){
			 				$setValue[]=$this->updateSourceAndReturnId($field,$subvalue);
			 			}
			 		}

			 	}
			 	/**This is the case of single select**/

			 	else {
			 		$setValue = null;
			 		$newOptionLabelArray=array();
			 		foreach ($options as $item) {
			 			if ($item['label'] == $value) {
			 				$setValue = $item['value'];
			 				array_push($optionLabelArray,$item['label']); /**Adding Reference to worked attribute option**/
			 			}
			 		}
			 		/**
			 		 *  Checking in the current XML column value if all values were used in the above loop or not
			 		 *  If not used then they are new options value, then new option is created and then assigned.
			 		 **/
			 		if(!in_array($value,$optionLabelArray)){
			 			$setValue=$this->updateSourceAndReturnId($field,$value);
			 		}
			 	}
			 }
			}

			/**CODE MODIFICATION ENDS HERE*/

			$product->setData($field, $setValue);
		}

		if (!$product->getVisibility()) {
			$product->setVisibility(Mage_Catalog_Model_Product_Visibility::VISIBILITY_NOT_VISIBLE);
		}

		$stockData = array();
		$inventoryFields = isset($this->_inventoryFieldsProductTypes[$product->getTypeId()])
		? $this->_inventoryFieldsProductTypes[$product->getTypeId()]
		: array();
		foreach ($inventoryFields as $field) {
			if (isset($importData[$field])) {
				if (in_array($field, $this->_toNumber)) {
					$stockData[$field] = $this->getNumber($importData[$field]);
				}
				else {
					$stockData[$field] = $importData[$field];
				}
			}
		}
		$product->setStockData($stockData);

		$imageData = array();
		foreach ($this->_imageFields as $field) {
			if (!empty($importData[$field]) &amp;amp;amp;&amp;amp;amp; $importData[$field] != 'no_selection') {
				if (!isset($imageData[$importData[$field]])) {
					$imageData[$importData[$field]] = array();
				}
				$imageData[$importData[$field]][] = $field;
			}
		}

		foreach ($imageData as $file => $fields) {
			try {
				$product->addImageToMediaGallery(Mage::getBaseDir('media') . DS . 'import' . $file, $fields);
			}
			catch (Exception $e) {}
		}

		$product->setIsMassupdate(true);
		$product->setExcludeUrlRewrite(true);
		$product->save();

		return true;
	}

	/**
	 * Updates the source of the attribute by the current new value in the XML column, and returns
	 * the id of the newly created option.
	 *
	 * @param string $attribute_code
	 * @param string $newOption
	 * @return int|string New Option Id
	 */
	public function updateSourceAndReturnId($attribute_code,$newOption){
		$attribute_model        = Mage::getModel('eav/entity_attribute');
		$attribute_options_model= Mage::getModel('eav/entity_attribute_source_table') ;

		$attribute_code         = $attribute_model->getIdByCode('catalog_product', $attribute_code);
		$attribute              = $attribute_model->load($attribute_code);
		$attribute_table        = $attribute_options_model->setAttribute($attribute);
		try{
			$value['option'] = array(trim($newOption),trim($newOption));
			$result = array('value' => $value);
			$attribute->setData('option',$result);
			$attribute->save();
		}
		catch(Exception $e){}
		$options = $attribute_options_model->getAllOptions(false);
		foreach($options as $option)
		{
			if ($option['label'] == $newOption)
			{
				return $option['value'];
			}
		}
		return "";
	}

}

?>

Please note the /**CODE MODIFICATION ENDS HERE*/ block. Most of the description is on the code. I got it working, feel free to ask if you got any problem. I am also working on importing custom option. I’ll soon make a post in it as well.

Download Source[Updated]

Here’s the sample import sheet with request from the reader

Sample Sheet