LSE Call Minutes 10/19/01 Attendees: Hubertus Franke, Ray Bryant, John Hawkes, Hanna Linder, Steve Carbonari, Rick Lindsley, Bill Irwin, Paul Jackson, Nick Polett, Jack Steiner, Paul Dorwin, Pat Gaughen, Tony Luck, Paul McKenney, Martin Bligh, Jack Vogel, John Stultz, Ruth Forester, Shailabh Nagar, John Wright ----- I. Hubertus Franke - Pipe expansion effect on benchmarks also compare with Manfred Spraul's zerocopy patch. Used three benchmarks lmbench, grep, pipeflex. See his email to lse-tech for more info. Showed substantial performance benefit on with some benchmarks on some systems. But there was no conclusive evidence to determine the best pipe size in every situation. The zerocopy patch showed performance improvements on different benchmarks often complementing the weakness of the pipe expansion patches. Q(?)- Are both (pipe extension and zerocopy) patches compatible? A(HF)- They are different approaches but they could be made compatible. Q(Jack V?)- Are you going to try dynamically changing things? A(HF)- We are going to try that. Maybe with a timing protocal. One problem with that is we will have to steal the memory at some point. We might be able to go to a filesystem to figure out which pipes are not busy then take their pages away... Comments by HF: The uniprocessor will not be effected by the pipe expansion patch because the way it is coded only effects smp systems. ---- II. Paul Jackson- CpuMemSets Summary: Cpumemsets arose out of looking at memory and proccessor affinity we had on Irix systems that we wanted to provide to customers on Linux side. It seems to me SGI has been more focused on static side whereas Sequent(now part of IBM) have been more focused on dynamic side. Cpumemsets is my proposal to do that. We are still working on the design. Additional capabilities I suspect will be needed are bulk operations. Which I define as operations that can apply to a number of procceses or memory sets at once. Need ability to go in kernel and say anything on cpu x move to cpu y. Move them all in one shot. ISSUES: Issue 1- placement policy Issue 2- do dealing with complicated mem list structure? Issue 3- support non root apps being able to subset their map to a smaller se to fthe map? and maintain it' Issue 4- origin of simple binding proposal and its motivations? Issue 1- Is it sufficient to have memory placement policy applied per object? or do we need placement policy down to pages? There is somthing at SGI called dplace. If you have an application that wants to run on a different number of proccesors than have in system then dplace allows you to run that app more efficiently without recompiling (it is a library?). Q(Paul Jackson)- What is the motivation of applying the placement policy per page? what applications use that? A(Paul McKenney)- Might see an address range where a data structure is placed might want to force them to be near each other. Another motivation is if you have large memory regions and want those objects pinned. Another case - if thing that touched it wants it close but being touch by two nodes need to do it explicitally there. Q(PJ) - Sounds like some of you see a need for per page memory region allocation? A(PMK)- At the march 23 meeting some people thought that was a good way to approach it. But there will be many different view points and it will not be trivial to resolve. Minutes are on the lse web page. Opinions might have changed based on new vm system in linux kernel... *AI = Someone is going to look into those minutes. Issue 4- PMK: Alot of back and forth at march 23 meeting. Simple binding came out as the least controversial. there is no installed base for it yet. Q(PMK)- one question I had what are the semantics of the memory portion of cms that are applied to cms current or child? What pieces of memory are those applied to? The reason I'm suspicious is because it seems cms current is used for nothing and cms child for two things. Might be more intuitive if current was for current and child for child. Q(PJ)- Clarify which two things? A(PMK)- 1) children processes (where their memory is taken from) 2) new memory object this process creates. It might be more natural if cms current specified where new object would be and cms child only specified where memory is taken from. A(PJ)- Ahhh. I think we have talked about this before but no one has ever explained it so well before. Thank you. Issue 3- Should non-root applications be able to map a subset? My hope was this is where simpler API's came in. PMK- Clearly we don't want non-root applications opening up the binding. III. Shailabh Just sent out patch where any io request is broken into three parts. People haven't had a chance to look at it. * Ruth offered to test it. IV. Other Jonh Wright will let us know when AIM is closer to being gpled. imminent within weeks. (from caldera).