Atom pipeline model, tuning and insn selection.

2009-04-06  Joey Ye  <joey.ye@intel.com>
	    Xuepeng Guo <xuepeng.guo@intel.com>
	    H.J. Lu  <hongjiu.lu@intel.com>

	Atom pipeline model, tuning and insn selection.
	* config.gcc (atom): Add atom config options and target.

	* config/i386/atom.md: New.

	* config/i386/i386.c (atom_cost): New cost.
	(m_ATOM): New macro flag.
	(initial_ix86_tune_features): Set m_ATOM.
	(x86_accumulate_outgoing_args): Likewise.
	(x86_arch_always_fancy_math_387): Likewise.
	(processor_target): Add Atom cost.
	(cpu_names): Add Atom cpu name.
	(override_options): Set Atom ISA.
	(ix86_issue_rate): New case PROCESSOR_ATOM.
	(ix86_adjust_cost): Likewise.

	* config/i386/i386.h (TARGET_ATOM): New target macro.
	(ix86_tune_indices): Add X86_TUNE_OPT_AGU.
	(TARGET_OPT_AGU): New target option.
	(target_cpu_default): Add TARGET_CPU_DEFAULT_atom.
	(processor_type): Add PROCESSOR_ATOM.

	* config/i386/i386.md (cpu): Add new value "atom".
	(use_carry, movu): New attr.
	(atom.md): Include atom.md.
	(adddi3_carry_rex64): Set attr "use_carry".
	(addqi3_carry): Likewise.
	(addhi3_carry): Likewise.
	(addsi3_carry): Likewise.
	(*addsi3_carry_zext): Likewise.
	(subdi3_carry_rex64): Likewise.
	(subqi3_carry): Likewise.
	(subhi3_carry): Likewise.
	(subsi3_carry): Likewise.
	(x86_movdicc_0_m1_rex64): Likewise.
	(*x86_movdicc_0_m1_se): Likewise.
	(x86_movsicc_0_m1): Likewise.
	(*x86_movsicc_0_m1_se): Likewise.
	(*adddi_1_rex64): Emit add insn as much as possible.
	(*addsi_1): Likewise.
	(return_internal): Set atom_unit.
	(return_internal_long): Likewise.
	(return_pop_internal): Likewise.
	(*rcpsf2_sse): Set atom_sse_attr attr.
	(*qrt<mode>2_sse): Likewise.
	(*prefetch_sse): Likewise.

	* config/i386/i386-c.c (ix86_target_macros_internal): New case
	PROCESSOR_ATOM.
	(ix86_target_macros_internal): Likewise.

	* config/i386/sse.md (cpu): Set attr "atom_sse_attr".
	(*prefetch_sse_rex): Likewise.
	(sse_rcpv4sf2): Likewise.
	(sse_vmrcpv4sf2): Likewise.
	(sse_sqrtv4sf2): Likewise.
	(<sse>_vmsqrt<mode>2): Likewise.
	(sse_ldmxcsr): Likewise.
	(sse_stmxcsr): Likewise.
	(*sse_sfence): Likewise.
	(sse2_clflush): Likewise.
	(*sse2_mfence): Likewise.
	(*sse2_lfence): Likewise.
	(avx_movup<avxmodesuffixf2c><avxmodesuffix>): Set attr "movu".
	(<sse>_movup<ssemodesuffixf2c>): Likewise.
	(avx_movdqu<avxmodesuffix>): Likewise.
	(avx_lddqu<avxmodesuffix>): Likewise.
	(sse2_movntv2di): Change attr "type" to "ssemov".
	(sse2_movntsi): Likewise.
	(rsqrtv8sf2): Change attr "type" to "sseadd".
	(sse3_addsubv2df3): Set attr "atom_unit".
	(sse3_h<plusminus_insn>v4sf3): Likewise.
	(*sse2_pmaddwd): Likewise.
	(*vec_extractv2di_1_rex64): Likewise.
	(*vec_extractv2di_1_avx): Likewise.
	(sse2_psadbw): Likewise.
	(ssse3_phaddwv8hi3): Likewise.
	(ssse3_phaddwv4hi3): Likewise.
	(ssse3_phadddv4si3): Likewise.
	(ssse3_phadddv2si3): Likewise.
	(ssse3_phaddswv8hi3): Likewise.
	(ssse3_phaddswv4hi3): Likewise.
	(ssse3_phsubwv8hi3): Likewise.
	(ssse3_phsubwv4hi3): Likewise.
	(ssse3_phsubdv4si3): Likewise.
	(ssse3_phsubdv2si3): Likewise.
	(ssse3_phsubswv8hi3): Likewise.
	(ssse3_phsubswv4hi3): Likewise.
	(ssse3_pmaddubsw128): Likewise.
	(sse3_pmaddubsw: Likewise.
	(ssse3_palignrti): Likewise.
	(ssse3_palignrdi): Likewise.

Co-Authored-By: H.J. Lu <hongjiu.lu@intel.com>
Co-Authored-By: Xuepeng Guo <xuepeng.guo@intel.com>

From-SVN: r145624

gcc/ChangeLog

+2009-04-06  Joey Ye  <joey.ye@intel.com>
+	    Xuepeng Guo <xuepeng.guo@intel.com>
+	    H.J. Lu  <hongjiu.lu@intel.com>
+
+	Atom pipeline model, tuning and insn selection.
+	* config.gcc (atom): Add atom config options and target.
+
+	* config/i386/atom.md: New.
+
+	* config/i386/i386.c (atom_cost): New cost.
+	(m_ATOM): New macro flag.
+	(initial_ix86_tune_features): Set m_ATOM.
+	(x86_accumulate_outgoing_args): Likewise.
+	(x86_arch_always_fancy_math_387): Likewise.
+	(processor_target): Add Atom cost.
+	(cpu_names): Add Atom cpu name.
+	(override_options): Set Atom ISA.
+	(ix86_issue_rate): New case PROCESSOR_ATOM.
+	(ix86_adjust_cost): Likewise.
+
+	* config/i386/i386.h (TARGET_ATOM): New target macro.
+	(ix86_tune_indices): Add X86_TUNE_OPT_AGU.
+	(TARGET_OPT_AGU): New target option.
+	(target_cpu_default): Add TARGET_CPU_DEFAULT_atom.
+	(processor_type): Add PROCESSOR_ATOM.
+
+	* config/i386/i386.md (cpu): Add new value "atom".
+	(use_carry, movu): New attr.
+	(atom.md): Include atom.md.
+	(adddi3_carry_rex64): Set attr "use_carry".
+	(addqi3_carry): Likewise.
+	(addhi3_carry): Likewise.
+	(addsi3_carry): Likewise.
+	(*addsi3_carry_zext): Likewise.
+	(subdi3_carry_rex64): Likewise.
+	(subqi3_carry): Likewise.
+	(subhi3_carry): Likewise.
+	(subsi3_carry): Likewise.
+	(x86_movdicc_0_m1_rex64): Likewise.
+	(*x86_movdicc_0_m1_se): Likewise.
+	(x86_movsicc_0_m1): Likewise.
+	(*x86_movsicc_0_m1_se): Likewise.
+	(*adddi_1_rex64): Emit add insn as much as possible.
+	(*addsi_1): Likewise.
+	(return_internal): Set atom_unit.
+	(return_internal_long): Likewise.
+	(return_pop_internal): Likewise.
+	(*rcpsf2_sse): Set atom_sse_attr attr.
+	(*qrt<mode>2_sse): Likewise.
+	(*prefetch_sse): Likewise.
+
+	* config/i386/i386-c.c (ix86_target_macros_internal): New case
+	PROCESSOR_ATOM.
+	(ix86_target_macros_internal): Likewise.
+
+	* config/i386/sse.md (cpu): Set attr "atom_sse_attr".
+	(*prefetch_sse_rex): Likewise.
+	(sse_rcpv4sf2): Likewise.
+	(sse_vmrcpv4sf2): Likewise.
+	(sse_sqrtv4sf2): Likewise.
+	(<sse>_vmsqrt<mode>2): Likewise.
+	(sse_ldmxcsr): Likewise.
+	(sse_stmxcsr): Likewise.
+	(*sse_sfence): Likewise.
+	(sse2_clflush): Likewise.
+	(*sse2_mfence): Likewise.
+	(*sse2_lfence): Likewise.
+	(avx_movup<avxmodesuffixf2c><avxmodesuffix>): Set attr "movu".
+	(<sse>_movup<ssemodesuffixf2c>): Likewise.
+	(avx_movdqu<avxmodesuffix>): Likewise.
+	(avx_lddqu<avxmodesuffix>): Likewise.
+	(sse2_movntv2di): Change attr "type" to "ssemov".
+	(sse2_movntsi): Likewise.
+	(rsqrtv8sf2): Change attr "type" to "sseadd".
+	(sse3_addsubv2df3): Set attr "atom_unit".
+	(sse3_h<plusminus_insn>v4sf3): Likewise.
+	(*sse2_pmaddwd): Likewise.
+	(*vec_extractv2di_1_rex64): Likewise.
+	(*vec_extractv2di_1_avx): Likewise.
+	(sse2_psadbw): Likewise.
+	(ssse3_phaddwv8hi3): Likewise.
+	(ssse3_phaddwv4hi3): Likewise.
+	(ssse3_phadddv4si3): Likewise.
+	(ssse3_phadddv2si3): Likewise.
+	(ssse3_phaddswv8hi3): Likewise.
+	(ssse3_phaddswv4hi3): Likewise.
+	(ssse3_phsubwv8hi3): Likewise.
+	(ssse3_phsubwv4hi3): Likewise.
+	(ssse3_phsubdv4si3): Likewise.
+	(ssse3_phsubdv2si3): Likewise.
+	(ssse3_phsubswv8hi3): Likewise.
+	(ssse3_phsubswv4hi3): Likewise.
+	(ssse3_pmaddubsw128): Likewise.
+	(sse3_pmaddubsw: Likewise.
+	(ssse3_palignrti): Likewise.
+	(ssse3_palignrdi): Likewise.
+
 2009-04-06  Gerald Pfeifer  <gerald@pfeifer.com>
 
 	* doc/install.texi (Specific): Fix two cross-references to MinGW.

gcc/config.gcc

@@ -1074,7 +1074,7 @@ i[34567]86-*-linux* | i[34567]86-*-kfreebsd*-gnu | i[34567]86-*-knetbsd*-gnu | i
 			tmake_file="${tmake_file} i386/t-linux64"
 			need_64bit_hwint=yes
 			case X"${with_cpu}" in
-			Xgeneric|Xcore2|Xnocona|Xx86-64|Xamdfam10|Xbarcelona|Xk8|Xopteron|Xathlon64|Xathlon-fx)
+			Xgeneric|Xatom|Xcore2|Xnocona|Xx86-64|Xamdfam10|Xbarcelona|Xk8|Xopteron|Xathlon64|Xathlon-fx)
 				;;
 			X)
 				if test x$with_cpu_64 = x; then
@@ -1083,7 +1083,7 @@ i[34567]86-*-linux* | i[34567]86-*-kfreebsd*-gnu | i[34567]86-*-knetbsd*-gnu | i
 				;;
 			*)
 				echo "Unsupported CPU used in --with-cpu=$with_cpu, supported values:" 1>&2
-				echo "generic core2 nocona x86-64 amdfam10 barcelona k8 opteron athlon64 athlon-fx" 1>&2
+				echo "generic atom core2 nocona x86-64 amdfam10 barcelona k8 opteron athlon64 athlon-fx" 1>&2
 				exit 1
 				;;
 			esac
@@ -1189,7 +1189,7 @@ i[34567]86-*-solaris2*)
 		need_64bit_hwint=yes
 		use_gcc_stdint=wrap
 		case X"${with_cpu}" in
-		Xgeneric|Xcore2|Xnocona|Xx86-64|Xamdfam10|Xbarcelona|Xk8|Xopteron|Xathlon64|Xathlon-fx)
+		Xgeneric|Xatom|Xcore2|Xnocona|Xx86-64|Xamdfam10|Xbarcelona|Xk8|Xopteron|Xathlon64|Xathlon-fx)
 			;;
 		X)
 			if test x$with_cpu_64 = x; then
@@ -1198,7 +1198,7 @@ i[34567]86-*-solaris2*)
 			;;
 		*)
 			echo "Unsupported CPU used in --with-cpu=$with_cpu, supported values:" 1>&2
-			echo "generic core2 nocona x86-64 amdfam10 barcelona k8 opteron athlon64 athlon-fx" 1>&2
+			echo "generic atom core2 nocona x86-64 amdfam10 barcelona k8 opteron athlon64 athlon-fx" 1>&2
 			exit 1
 			;;
 		esac
@@ -2801,7 +2801,7 @@ case "${target}" in
 				esac
 				# OK
 				;;
-			"" | amdfam10 | barcelona | k8 | opteron | athlon64 | athlon-fx | nocona | core2 | generic)
+			"" | amdfam10 | barcelona | k8 | opteron | athlon64 | athlon-fx | nocona | core2 | atom | generic)
 				# OK
 				;;
 			*)

gcc/config/i386/i386-c.c

@@ -119,6 +119,10 @@ ix86_target_macros_internal (int isa_flag,
       def_or_undef (parse_in, "__core2");
       def_or_undef (parse_in, "__core2__");
       break;
+    case PROCESSOR_ATOM:
+      def_or_undef (parse_in, "__atom");
+      def_or_undef (parse_in, "__atom__");
+      break;
     /* use PROCESSOR_max to not set/unset the arch macro.  */
     case PROCESSOR_max:
       break;
@@ -187,6 +191,9 @@ ix86_target_macros_internal (int isa_flag,
     case PROCESSOR_CORE2:
       def_or_undef (parse_in, "__tune_core2__");
       break;
+    case PROCESSOR_ATOM:
+      def_or_undef (parse_in, "__tune_atom__");
+      break;
     case PROCESSOR_GENERIC32:
     case PROCESSOR_GENERIC64:
       break;

gcc/config/i386/i386.h

@@ -231,6 +231,7 @@ extern const struct processor_costs ix86_size_cost;
 #define TARGET_GENERIC64 (ix86_tune == PROCESSOR_GENERIC64)
 #define TARGET_GENERIC (TARGET_GENERIC32 || TARGET_GENERIC64)
 #define TARGET_AMDFAM10 (ix86_tune == PROCESSOR_AMDFAM10)
+#define TARGET_ATOM (ix86_tune == PROCESSOR_ATOM)
 
 /* Feature tests against the various tunings.  */
 enum ix86_tune_indices {
@@ -295,6 +296,7 @@ enum ix86_tune_indices {
   X86_TUNE_USE_VECTOR_FP_CONVERTS,
   X86_TUNE_USE_VECTOR_CONVERTS,
   X86_TUNE_FUSE_CMP_AND_BRANCH,
+  X86_TUNE_OPT_AGU,
 
   X86_TUNE_LAST
 };
@@ -382,6 +384,7 @@ extern unsigned char ix86_tune_features[X86_TUNE_LAST];
 	ix86_tune_features[X86_TUNE_USE_VECTOR_CONVERTS]
 #define TARGET_FUSE_CMP_AND_BRANCH \
 	ix86_tune_features[X86_TUNE_FUSE_CMP_AND_BRANCH]
+#define TARGET_OPT_AGU ix86_tune_features[X86_TUNE_OPT_AGU]
 
 /* Feature tests against the various architecture variations.  */
 enum ix86_arch_indices {
@@ -567,6 +570,7 @@ enum target_cpu_default
   TARGET_CPU_DEFAULT_prescott,
   TARGET_CPU_DEFAULT_nocona,
   TARGET_CPU_DEFAULT_core2,
+  TARGET_CPU_DEFAULT_atom,
 
   TARGET_CPU_DEFAULT_geode,
   TARGET_CPU_DEFAULT_k6,
@@ -2272,6 +2276,7 @@ enum processor_type
   PROCESSOR_GENERIC32,
   PROCESSOR_GENERIC64,
   PROCESSOR_AMDFAM10,
+  PROCESSOR_ATOM,
   PROCESSOR_max
 };
 

gcc/config/i386/i386.md

@@ -316,7 +316,7 @@
 
 
 ;; Processor type.
-(define_attr "cpu" "none,pentium,pentiumpro,geode,k6,athlon,k8,core2,
+(define_attr "cpu" "none,pentium,pentiumpro,geode,k6,athlon,k8,core2,atom,
 		    generic64,amdfam10"
   (const (symbol_ref "ix86_schedule")))
 
@@ -612,6 +612,12 @@
 (define_attr "i387_cw" "trunc,floor,ceil,mask_pm,uninitialized,any"
   (const_string "any"))
 
+;; Define attribute to classify add/sub insns that consumes carry flag (CF)
+(define_attr "use_carry" "0,1" (const_string "0"))
+
+;; Define attribute to indicate unaligned ssemov insns
+(define_attr "movu" "0,1" (const_string "0"))
+
 ;; Describe a user's asm statement.
 (define_asm_attributes
   [(set_attr "length" "128")
@@ -727,6 +733,7 @@
 (include "k6.md")
 (include "athlon.md")
 (include "geode.md")
+(include "atom.md")
 
 
 ;; Operand and operator predicates and constraints
@@ -5816,6 +5823,7 @@
   "TARGET_64BIT && ix86_binary_operator_ok (PLUS, DImode, operands)"
   "adc{q}\t{%2, %0|%0, %2}"
   [(set_attr "type" "alu")
+   (set_attr "use_carry" "1")
    (set_attr "pent_pair" "pu")
    (set_attr "mode" "DI")])
 
@@ -5890,6 +5898,7 @@
   "ix86_binary_operator_ok (PLUS, QImode, operands)"
   "adc{b}\t{%2, %0|%0, %2}"
   [(set_attr "type" "alu")
+   (set_attr "use_carry" "1")
    (set_attr "pent_pair" "pu")
    (set_attr "mode" "QI")])
 
@@ -5902,6 +5911,7 @@
   "ix86_binary_operator_ok (PLUS, HImode, operands)"
   "adc{w}\t{%2, %0|%0, %2}"
   [(set_attr "type" "alu")
+   (set_attr "use_carry" "1")
    (set_attr "pent_pair" "pu")
    (set_attr "mode" "HI")])
 
@@ -5914,6 +5924,7 @@
   "ix86_binary_operator_ok (PLUS, SImode, operands)"
   "adc{l}\t{%2, %0|%0, %2}"
   [(set_attr "type" "alu")
+   (set_attr "use_carry" "1")
    (set_attr "pent_pair" "pu")
    (set_attr "mode" "SI")])
 
@@ -5927,6 +5938,7 @@
   "TARGET_64BIT && ix86_binary_operator_ok (PLUS, SImode, operands)"
   "adc{l}\t{%2, %k0|%k0, %2}"
   [(set_attr "type" "alu")
+   (set_attr "use_carry" "1")
    (set_attr "pent_pair" "pu")
    (set_attr "mode" "SI")])
 
@@ -6156,9 +6168,9 @@
    (set_attr "mode" "SI")])
 
 (define_insn "*adddi_1_rex64"
-  [(set (match_operand:DI 0 "nonimmediate_operand" "=r,rm,r")
-	(plus:DI (match_operand:DI 1 "nonimmediate_operand" "%0,0,r")
-		 (match_operand:DI 2 "x86_64_general_operand" "rme,re,le")))
+  [(set (match_operand:DI 0 "nonimmediate_operand" "=r,rm,r,r")
+	(plus:DI (match_operand:DI 1 "nonimmediate_operand" "%0,0,r,r")
+		 (match_operand:DI 2 "x86_64_general_operand" "rme,re,0,le")))
    (clobber (reg:CC FLAGS_REG))]
   "TARGET_64BIT && ix86_binary_operator_ok (PLUS, DImode, operands)"
 {
@@ -6179,6 +6191,10 @@
 	}
 
     default:
+      /* Use add as much as possible to replace lea for AGU optimization. */
+      if (which_alternative == 2 && TARGET_OPT_AGU)
+        return "add{q}\t{%1, %0|%0, %1}";
+        
       gcc_assert (rtx_equal_p (operands[0], operands[1]));
 
       /* Make things pretty and `subl $4,%eax' rather than `addl $-4, %eax'.
@@ -6197,8 +6213,11 @@
     }
 }
   [(set (attr "type")
-     (cond [(eq_attr "alternative" "2")
+     (cond [(and (eq_attr "alternative" "2") 
+                 (eq (symbol_ref "TARGET_OPT_AGU") (const_int 0)))
 	      (const_string "lea")
+            (eq_attr "alternative" "3")
+              (const_string "lea")
 	    ; Current assemblers are broken and do not allow @GOTOFF in
 	    ; ought but a memory context.
 	    (match_operand:DI 2 "pic_symbolic_operand" "")
@@ -6215,8 +6234,7 @@
 	(plus:DI (match_operand:DI 1 "register_operand" "")
 		 (match_operand:DI 2 "x86_64_nonmemory_operand" "")))
    (clobber (reg:CC FLAGS_REG))]
-  "TARGET_64BIT && reload_completed
-   && true_regnum (operands[0]) != true_regnum (operands[1])"
+  "TARGET_64BIT && reload_completed"
   [(set (match_dup 0)
 	(plus:DI (match_dup 1)
 		 (match_dup 2)))]
@@ -6420,9 +6438,9 @@
 
 
 (define_insn "*addsi_1"
-  [(set (match_operand:SI 0 "nonimmediate_operand" "=r,rm,r")
-	(plus:SI (match_operand:SI 1 "nonimmediate_operand" "%0,0,r")
-		 (match_operand:SI 2 "general_operand" "g,ri,li")))
+  [(set (match_operand:SI 0 "nonimmediate_operand" "=r,rm,r,r")
+	(plus:SI (match_operand:SI 1 "nonimmediate_operand" "%0,0,r,r")
+		 (match_operand:SI 2 "general_operand" "g,ri,0,li")))
    (clobber (reg:CC FLAGS_REG))]
   "ix86_binary_operator_ok (PLUS, SImode, operands)"
 {
@@ -6443,6 +6461,10 @@
 	}
 
     default:
+      /* Use add as much as possible to replace lea for AGU optimization. */
+      if (which_alternative == 2 && TARGET_OPT_AGU)
+        return "add{l}\t{%1, %0|%0, %1}";
+
       gcc_assert (rtx_equal_p (operands[0], operands[1]));
 
       /* Make things pretty and `subl $4,%eax' rather than `addl $-4, %eax'.
@@ -6459,7 +6481,10 @@
     }
 }
   [(set (attr "type")
-     (cond [(eq_attr "alternative" "2")
+     (cond [(and (eq_attr "alternative" "2") 
+                 (eq (symbol_ref "TARGET_OPT_AGU") (const_int 0)))
+               (const_string "lea")
+            (eq_attr "alternative" "3")
 	      (const_string "lea")
 	    ; Current assemblers are broken and do not allow @GOTOFF in
 	    ; ought but a memory context.
@@ -6477,8 +6502,7 @@
 	(plus (match_operand 1 "register_operand" "")
               (match_operand 2 "nonmemory_operand" "")))
    (clobber (reg:CC FLAGS_REG))]
-  "reload_completed
-   && true_regnum (operands[0]) != true_regnum (operands[1])"
+  "reload_completed"
   [(const_int 0)]
 {
   rtx pat;
@@ -7580,6 +7604,7 @@
   "TARGET_64BIT && ix86_binary_operator_ok (MINUS, DImode, operands)"
   "sbb{q}\t{%2, %0|%0, %2}"
   [(set_attr "type" "alu")
+   (set_attr "use_carry" "1")
    (set_attr "pent_pair" "pu")
    (set_attr "mode" "DI")])
 
@@ -7628,6 +7653,7 @@
   "ix86_binary_operator_ok (MINUS, QImode, operands)"
   "sbb{b}\t{%2, %0|%0, %2}"
   [(set_attr "type" "alu")
+   (set_attr "use_carry" "1")
    (set_attr "pent_pair" "pu")
    (set_attr "mode" "QI")])
 
@@ -7640,6 +7666,7 @@
   "ix86_binary_operator_ok (MINUS, HImode, operands)"
   "sbb{w}\t{%2, %0|%0, %2}"
   [(set_attr "type" "alu")
+   (set_attr "use_carry" "1")
    (set_attr "pent_pair" "pu")
    (set_attr "mode" "HI")])
 
@@ -7652,6 +7679,7 @@
   "ix86_binary_operator_ok (MINUS, SImode, operands)"
   "sbb{l}\t{%2, %0|%0, %2}"
   [(set_attr "type" "alu")
+   (set_attr "use_carry" "1")
    (set_attr "pent_pair" "pu")
    (set_attr "mode" "SI")])
 
@@ -15275,6 +15303,7 @@
   "reload_completed"
   "ret"
   [(set_attr "length" "1")
+   (set_attr "atom_unit" "jeu")
    (set_attr "length_immediate" "0")
    (set_attr "modrm" "0")])
 
@@ -15287,6 +15316,7 @@
   "reload_completed"
   "rep\;ret"
   [(set_attr "length" "1")
+   (set_attr "atom_unit" "jeu")
    (set_attr "length_immediate" "0")
    (set_attr "prefix_rep" "1")
    (set_attr "modrm" "0")])
@@ -15297,6 +15327,7 @@
   "reload_completed"
   "ret\t%0"
   [(set_attr "length" "3")
+   (set_attr "atom_unit" "jeu")
    (set_attr "length_immediate" "2")
    (set_attr "modrm" "0")])
 
@@ -16418,6 +16449,7 @@
   "TARGET_SSE_MATH"
   "%vrcpss\t{%1, %d0|%d0, %1}"
   [(set_attr "type" "sse")
+   (set_attr "atom_sse_attr" "rcp")
    (set_attr "prefix" "maybe_vex")
    (set_attr "mode" "SF")])
 
@@ -16777,6 +16809,7 @@
   "TARGET_SSE_MATH"
   "%vrsqrtss\t{%1, %d0|%d0, %1}"
   [(set_attr "type" "sse")
+   (set_attr "atom_sse_attr" "rcp")
    (set_attr "prefix" "maybe_vex")
    (set_attr "mode" "SF")])
 
@@ -16797,6 +16830,7 @@
   "SSE_FLOAT_MODE_P (<MODE>mode) && TARGET_SSE_MATH"
   "%vsqrts<ssemodefsuffix>\t{%1, %d0|%d0, %1}"
   [(set_attr "type" "sse")
+   (set_attr "atom_sse_attr" "sqrt")
    (set_attr "prefix" "maybe_vex")
    (set_attr "mode" "<MODE>")
    (set_attr "athlon_decode" "*")
@@ -19850,6 +19884,7 @@
   ; Since we don't have the proper number of operands for an alu insn,
   ; fill in all the blanks.
   [(set_attr "type" "alu")
+   (set_attr "use_carry" "1")
    (set_attr "pent_pair" "pu")
    (set_attr "memory" "none")
    (set_attr "imm_disp" "false")
@@ -19865,6 +19900,7 @@
   ""
   "sbb{q}\t%0, %0"
   [(set_attr "type" "alu")
+   (set_attr "use_carry" "1")
    (set_attr "pent_pair" "pu")
    (set_attr "memory" "none")
    (set_attr "imm_disp" "false")
@@ -19908,6 +19944,7 @@
   ; Since we don't have the proper number of operands for an alu insn,
   ; fill in all the blanks.
   [(set_attr "type" "alu")
+   (set_attr "use_carry" "1")
    (set_attr "pent_pair" "pu")
    (set_attr "memory" "none")
    (set_attr "imm_disp" "false")
@@ -19923,6 +19960,7 @@
   ""
   "sbb{l}\t%0, %0"
   [(set_attr "type" "alu")
+   (set_attr "use_carry" "1")
    (set_attr "pent_pair" "pu")
    (set_attr "memory" "none")
    (set_attr "imm_disp" "false")
@@ -20255,7 +20293,8 @@
     }
 }
   [(set (attr "type")
-	(cond [(eq_attr "alternative" "0")
+	(cond [(and (eq_attr "alternative" "0") 
+	            (eq (symbol_ref "TARGET_OPT_AGU") (const_int 0)))
 		 (const_string "alu")
 	       (match_operand:SI 2 "const0_operand" "")
 		 (const_string "imov")
@@ -20298,7 +20337,8 @@
     }
 }
   [(set (attr "type")
-	(cond [(eq_attr "alternative" "0")
+	(cond [(and (eq_attr "alternative" "0")
+	            (eq (symbol_ref "TARGET_OPT_AGU") (const_int 0)))
 		 (const_string "alu")
 	       (match_operand:DI 2 "const0_operand" "")
 		 (const_string "imov")
@@ -21790,6 +21830,7 @@
   return patterns[locality];
 }
   [(set_attr "type" "sse")
+   (set_attr "atom_sse_attr" "prefetch")
    (set_attr "memory" "none")])
 
 (define_insn "*prefetch_sse_rex"
@@ -21808,6 +21849,7 @@
   return patterns[locality];
 }
   [(set_attr "type" "sse")
+   (set_attr "atom_sse_attr" "prefetch")
    (set_attr "memory" "none")])
 
 (define_insn "*prefetch_3dnow"