Minutes from 10/05 LSE con call


John Hawkes, Steve Carbonari, Jesse Barnes, Richard Griffith, Terry Prickett
Tony Luck, Jim Washer, Paul Jackson, Paul McKenney, Gerrit Huizenga,
Pat Gaughen, Jack Vogel, Hans Tannenberger, Bill Irwin, Paul Dorwin,
Janet Morgan, John Stultz, Matt Dobson, Johnathan Lahr, Martin Bligh,
Hanna Linder, Rick Lindsley, Jay Urbanksi, Jack Stiener, Bill Hartner


Summary:
Action Items specified with *

I. Paul McKenney- 
	See http://lse.sourceforge.net/numa/numa_api.html

	SGI did a proof of concept boot of 128 cpu machine
	almost a year ago. Martin Bligh asked to see their
	patch. 
    *Someone from SGI will make that available.

	Will be easier to get code into the kernel if we
	keep changes out of core/common code, try to keep
	changes on the side. That will part of the goal 
	with this designing.

	OK to have threads want to use different memory binding 
	policies, whichever one faults the page uses the
	binding they want. The others silently fail. Having
	differnt policies isnt proven yet to be either stupid
	or smart, neither seems provably better so lets try
	the simplest approach.

	Q(tony l)- Is it possible to extend so some non numa aware 
	application can run efficiently?  
	A(paul m)- I believe that SGI is working on that(dplace)?  
	If not then we will work on that because it is very
	important. There is a man page for dplace on 
	http://lse.sourceforge.net/numa/manpages
	Q to Tony- Are you referring to a running application 
	or one that is about to be launcched?
	A(tony) Either really.

    *(paul m) We need to have a way to control processes 
	that already exist and binding them one way or another.

	SGI guys really liked the HP launch (?) solution.

        Comment to Paul M: make cpu parameters clearer, if they
	are relative or absolute. 
    *(paul m) It is always absolute unles (?). Paul will
	make that more clear.

	
II. Pat Gaughen-
	Recently took over the numa status web page from Paul M.
	http://lse.sf.net/numa/numa_status.html

	If you are involved with NUMA look at it and if you
	are working on NUMA please keep her updated.
	Make sure your stuff is up to date.

III. Shailabh Nagar-
	Recently put out patch. Confirms still a benefit 
	on large systems (running with 4k atomic io size.)
	I think (bill h) we have a system set up with
	ddtest, bouce buffer, next is to try increasing
	block size to 4k. 
     *john tran at db2 to run some tests.

	(Shailabh n) We want to verify no filesystem
	creation or mounting breaks.  verify users of 
	same code path do not break.

	Does anyone have any concerns about including 
	atomic size in raw i/o access in general? 
      *will ask on lse tech.

IV. Hanna Linder-
	Mentioned the sorted results on lse.sf.net/locking
	for the dcache_lock patch are a little confusing
	because the function are not mapped to the
	locks they call. Have to look at the clean unsorted
	results to see that mapping. Asked John Hawkes if
	he would be able to add a sort option to lockstat
	that could deal with that problem.
    *John Hawkes said he would look into it (only a matter
	of programming ;)

	Hanna will do more testing next week using stat
	and tux (from a recommendation by Andrea Arcangeli).
	Results on 2.4.10 also available next week.

	John Hawkes said total spinlock contention of 10%
	is not bad at all for a big NUMA system.

	Gerrit Huizenga- Our goal is 75% utilizatin of all cpus.
	is that a goal for you?

	John Hawkes- That is one of the things we find important.
	Spinlocks have gotten much better in the last few months.
	What I see as the biggest problem is cache block ping-
	ponging and cache block write back.

	Q(gerrit h)- Is there a tool to look at that?
	A(john h)- Not one as clean as lockstat is for locks.
	Shailabh N- Would someone be interested in writing a 
	tool for this? It would be very helpful.

	Discusion of different tools that showed cache and tlb
	misses. see lse-tech mail list for some of those tools
	(papi, perfmeter...)

	John Hawkes- Those tools all show cache misses. But the
	most useful information is to know which code caused
	the cache misses.

	SGI had a tool called pixie that did this. It showed
	all blocks in the kernel and referred to them by
	memory address and symbolically by function name
	and line. Involved compiler changes.


V. Ruth Forester-
     *As soon she sees aim7 is publically available she will
	email the list.