Ubuntu 11 Drillbit process timeout issue

[12:46pm] dubcanada_: I suspect I'll probably get it tonight
[12:46pm] dubcanada_: I have a bunch of ideas to try.
[12:47pm] fairwinds: dubcanada_: I think I will contact marshal today also. I see him on titanium_app. He is very helpful and knows more
[12:47pm] dubcanada_: I just need someone to bounce things off
[12:48pm] fairwinds: dubcanada_: are you here for a bit. I will see if marshall can join us here
[12:49pm] fairwinds: k super, also we have  gists somewhere on this  issue can you point me to these right now
[12:49pm] fairwinds: dubcanada_:
[12:49pm] dubcanada_: Ummm gists for Ubuntu 11?
[12:50pm] fairwinds: for the process issue on drill bit
[12:50pm] dubcanada_: ah okay
[12:50pm] dubcanada_: I rewrote the drill bits set interval function with https://gist.github.com/b706bab5870eb6a51157
[12:50pm] dubcanada_: drillbitsmodule.js file
[12:51pm] dubcanada_: The rest why we we're having problems is when we started Drillbits it worked fine
[12:51pm] dubcanada_: When we ran a test it always died with the undefined string:555 error
[12:51pm] fairwinds: sure, that around line 550
[12:51pm] dubcanada_: test.current_test.failed
[12:51pm] dubcanada_: Yes
[12:51pm] fairwinds: k
[12:52pm] dubcanada_: After investigating I determined that function was not updated
[12:52pm] dubcanada_: (This function does not happen on other OS's)
[12:52pm] dubcanada_: because the other OS's don't time out
[12:52pm] fairwinds: right
[12:52pm] dubcanada_: The real issue is the tests are timing out for Ubuntu 11
[12:52pm] dubcanada_: And I wasn't sure if that was a VM issue or not.
[12:52pm] dubcanada_: They really stupidly slow
[12:52pm] dubcanada_: Like it took 5 minutes to run the Ajax test
[12:52pm] fairwinds: blackorzar_: ran into these issues as well
[12:52pm] dubcanada_: Yes
[12:53pm] dubcanada_: As did Daegalus
[12:53pm] dubcanada_: So I suspect it's not a VM issue
[12:53pm] fairwinds: nor do i
[12:53pm] blackorzar_: dubcanada_: nice!!!
[12:53pm] dubcanada_: So it's one of two things.
[12:53pm] dubcanada_: 1. There is an issue with Webkit.
[12:53pm] dubcanada_: 2. There is an issue somewhere in the Drillbits JS files
[12:53pm] blackorzar_: agreed
[12:54pm] dubcanada_: Something that would be causing it to timeout.
[12:54pm] dubcanada_: Something it causing it to run extremely slow.
[12:54pm] dubcanada_: I suspect it has to do with the Webkit
[12:54pm] dubcanada_: as each OS has a different webkit version.
[12:54pm] blackorzar_: dubcanada_: how do you think it would be possible to debug that?
[12:54pm] dubcanada_: Well I'd really like to start looking at upgrading Webkit
[12:54pm] dubcanada_: In regards to how to debug that I will need to continue playing with it.
[12:55pm] dubcanada_: I really wish I had V7
[12:55pm] dubcanada_: V8*
[12:55pm] dubcanada_: Cause I could use console logs to do performance stuff
[12:55pm] fairwinds: blackorzar_: marshall is on the #titanium_app channel today. going to see if he can jump in and discuss this issue with us
[12:56pm] fairwinds: blackorzar_: are you around for a bit if he is availabe
[12:56pm] fairwinds: dubcanada_: yes, absolute
[12:57pm] dubcanada__ joined the chat room.
[12:57pm] dubcanada__: Did you get my msg's?
[12:57pm] blackorzar_: wow i didn't knew that
[12:57pm] blackorzar_: maybe mrobinson can give us a clue here
[12:57pm] dubcanada__: yah
[12:58pm] dubcanada__: I'd really like to upgrade webkit
[12:58pm] dubcanada__: I think that would solve a ton of issues
[12:58pm] dubcanada__: Even if we can maybe bring it up to 6 months old
[12:58pm] dubcanada__: Ubuntu 12 and Ubuntu 11 would most likely start working
[12:58pm] blackorzar_: fairwinds: yeap
[12:58pm] blackorzar_: dubcanada_: agreed
[12:59pm] dubcanada__: Most of the webkit patches are simply commenting out security features
[12:59pm] dubcanada__: Sadly I wish I had a PatchLog file with a list of it.
[12:59pm] dubcanada__: I could most likely do it in a day.
[12:59pm] blackorzar_: wow
[12:59pm] blackorzar_: nice
[1:00pm] fairwinds: yes, this is what I will ask marshall about as well, I did ask him for documentation on this when we had our scheduled q & a
[1:00pm] dubcanada_ left the chat room. (Ping timeout: 272 seconds)
[1:01pm] fairwinds: dubcanada_: worse case I will see if marshall can give us some scheduled time if he cannot just jump in
[1:02pm] fairwinds: but he is very supportive of what we are doing here and a great guy also
[1:03pm] matthewh: fairwinds: lion is up and running. installing java now. almost connected to jenkins...
[1:03pm] dubcanada_: The Merge WebKit upstream will be extremely difficult
[1:03pm] fairwinds: awesome!
[1:04pm] matthewh: (boot time for lion VM is 8 minutes)
[1:04pm] fairwinds: wow
[1:04pm] dubcanada__ left the chat room. (Ping timeout: 264 seconds)
[1:04pm] dubcanada_: Last one joshthecoder did was 11,528 changed files, 273,161 additions and 79369 deletions
[1:04pm] dubcanada_: So that will be fun to go through
[1:04pm] dubcanada_: And that was only about 4-5 months old
[1:05pm] fairwinds: dubcanada_: can you paste the who drillbitmodule.py somewhere
[1:05pm] fairwinds: for linux
[1:05pm] fairwinds: that you have there
[1:05pm] dubcanada_: I just crashed my browser trying to open that merge commit on github
[1:06pm] dubcanada_: lol
[1:06pm] fairwinds: heh
[1:07pm] fairwinds: s/ whole file. not who 
[1:09pm] dubcanada_: https://gist.github.com/4e46cb4084897ab0cb26
[1:11pm] fairwinds: blackorzar: what versions of ubuntu did you build specifically
[1:11pm] blackorzar: 11.04
[1:11pm] dubcanada_: 11.10
[1:12pm] dubcanada_: To be specific
[1:12pm] fairwinds: k
[1:12pm] dubcanada_: One Oneric
[1:12pm] dubcanada_: In order to get past a few of the errors
[1:12pm] dubcanada_: You need to install the natty repo
[1:12pm] dubcanada_: and apt-get install libnotify1 libffi
[1:13pm] dubcanada_: Which will solve the Missing libffi.so.5 error
[1:13pm] fairwinds: k, thats important. please make note of this somewhere
[1:14pm] fairwinds: blackorzar: that did not seem to be necessary on your build I recall
[1:15pm] blackorzar: that's right
[1:15pm] fairwinds: k, just wanted to verify

 issues and possibilities and perhaps what more we can do to debug this
[2:20pm] dubcanada_: Okay well
[2:25pm] blackorzar: Windows and Linux tests are going really slow... not sure if that's normal or something related with webkit/drillbit, and if you can give us a tip to start making the profiling would be great
[2:26pm] blackorzar: marshall_law: maybe some debug logs , or like you said adding an async test... not sure how to approach this one
[2:27pm] fairwinds: marshall_law: what we have observed is windows test run almost 10x slower than macosx tests and on linux when we get to tests there it will hang. dubcanada has spent some time attempting to isolate the issue
[2:27pm] fairwinds: but we may need some hints on what more we might do to analyse his problem and resolve it
[2:29pm] dubcanada_: Yah sorry I lost internet
[2:29pm] dubcanada_: I got the messages
[2:29pm] fairwinds: What is different about linux compared to the other os's thus far is that it is entering this stuck process monitor
[2:29pm] dubcanada_: Yah that's pretty much the main problem
[2:29pm] meeech: fairwinds: was just searching through transcript
[2:30pm] fairwinds: and hanging there. this is around line 450 of the drillbitmodule.py
[2:30pm] meeech: you wanted to know about why i used tidesdk?
[2:30pm] fairwinds: meeech: yup. can I ping you back in a bout 30 min on that
[2:30pm] meeech: kk. i should be around, just going for coffee.
[2:31pm] fairwinds: sure
[2:32pm] marshall_law: IIRC there was some problem with process I/O and python's subprocess API. I don't remember details but it had something to do with python blocking on the process buffer not being full
[2:32pm] marshall_law: could be that process.exit() is hanging in python waiting for more input
[2:33pm] dubcanada_: One said to note is we've been using 2.7 Python for everything
[2:33pm] dubcanada_: One thing*
[2:33pm] dubcanada_: So would it be save to assume this is most likely a Python issue/python script/javascript combination?
[2:34pm] dubcanada_: And do you know if there was any ability to do any performance calculations (something like console.log) in Drillbit? That would help a lot.
[2:37pm] fairwinds: dub because drillbit an app itself we could look at Titanium.API.info possibly
[2:37pm] fairwinds: dubcanada_:
[2:37pm] fairwinds: at least inside the app
[2:37pm] dubcanada_: Does .info allow you to post to console?
[2:38pm] dubcanada_: I need some way to go through and determine what is slowing things down
[2:39pm] fairwinds: It does since this output was just captured for TiDeveloper after process started
[2:39pm] fairwinds: and displayed through the app
[2:41pm] fairwinds: If we use debug perhasp we'll also get some good feedback through the app
[2:42pm] dubcanada_: hrm
[2:42pm] fairwinds: but I think we need to be looking at this a bit from the outside also
[2:43pm] fairwinds: as to what from python
[2:43pm] fairwinds: might be affecting this
[2:44pm] fairwinds: dubcanada_: can we possibly drop back on python to try this
[2:44pm] dubcanada_: 2.6?
[2:44pm] fairwinds: yes
[2:44pm] dubcanada_: I'll give it a try
[2:44pm] fairwinds: k, super
[2:45pm] fairwinds: just to see if any change
[2:45pm] fairwinds: dubcanada: also you mentioned you had some other ideas to try, what were your thoughts
[2:46pm] fairwinds: we also have python debugger to perhaps step through some break points
[2:46pm] fairwinds: and try to see what python is up to
[2:47pm] fairwinds: this might help identify the sort of issue marshall has identified
[2:47pm] fairwinds: as a possibility
[2:48pm] fairwinds: so that is something that can be done directly in that file
[2:48pm] fairwinds: the drillbitmodule.py
[2:49pm] fairwinds: dubcanada_, blackorzar: your thoughts on other things we might try?
[2:49pm] dubcanada_: Well I didn't know about the python issues
[2:49pm] dubcanada_: So that kind of throws out a lot of ideas I had. Which where mostly related to javascript
[2:50pm] dubcanada_: One of the things I'm wondering right now is still the Webkit version
[2:50pm] blackorzar: hmmm dubcanada_ have you tried another app (not a drillbit one)?
[2:50pm] blackorzar: i think josh or marshall had a benchmark
[2:50pm] blackorzar: on their repos
[2:50pm] dubcanada_: Where?
[2:50pm] blackorzar: or martin
[2:51pm] blackorzar: let me check
[2:51pm] blackorzar: brb
[2:52pm] fairwinds: dubcanada_:  I think we can't immediately assume python but it is a place to look. I think what blackorzar might be thinking is we try launching an app with the sdk and see what happens
[2:52pm] blackorzar: https://github.com/TideSDK/titanium_json_benchmark
[2:53pm] blackorzar: not sure if it would be helpful
[2:53pm] fairwinds: it seems process related so it is a good thing to examine
[2:54pm] fairwinds: blackorzar: you pull these things out of a hat, awesome 
[2:55pm] blackorzar: jaja doing a lot of research
[2:55pm] fairwinds: yeah, v good
[2:55pm] fairwinds: dubcanada: on the js side, what were you thinking to try?