Report on new optimized Astropulse apps for Windows

[Jason G]

A ship.  It supposedly was supposed to get a wartime transmission/order that was received by it's successor, Queen Mary 2 Elizabeth II, (~35 years? later )... basically the point is that radio waves do weird stuff, and that messing with a single sample isn't likely to upset the kind of transmission we're looking for, especially when you're talking broadband pulses that saturate surrounding frequencies... (Will check name of etc.... Though that sounds about right  )

[Edit:  intended recipient was the 'Queen Mary' during world war II, recipient was the  QE2 , "Queen Elizabeth the 2nd" in 1978... No subscription by myself about paranormal factors or ' Blackwell Bracewell probe' theories either, but can attest that radio waves do weird stuff  anyway ]

[Josef W. Segur]

@All testers
Please keep eye on indices.txt in working directory (if it become non-zero length or not).
And if you keep result file, please, keep indices.txt too along with result and WU files.

ap_03no08af_B2_P1_00143_20081121_30048 (1065721003)
ap_03no08ag_B2_P1_00100_20081121_20392 (1066329522)

- both just started - have chunky indices.txt files (attached). If I'm reading the guesstimates right, these both have a high chance of pulsing out before their allotted span. Have suspended networking for a result capture.

That second one is a jewel. 2141 blanking indices, leaves about 6422528 samples to process so roughly 81% is blanked. A waste of CPU, but intellectually interesting. If it exits early I suspect the percent blanking reported in the result file won't agree with my calculations, it's done incrementally. In theory it should not be likely to exit early, in practice I'm not even willing to guess.

The indices.txt file isn't used for anything, just written during the initialization step which generates the internal array. It's basically the same information which was put into stderr.txt by the Linux 4.37 stock build, but BOINC will only report the last 64K of stderr. I'm glad they preserved it in this form even if it is ridiculously verbose.
                                                                       Joe

[Raistmer]

We can use that file for determining blanked areas and to see if these blanked areas gave some signals or not....

[Jason G]

We can use that file for determining blanked areas and to see if these blanked areas gave some signals or not....

  I should hope so, if there's no signal there then there was no point blanking it right?

[Raistmer]

After blanking there should be NO signal. OR this blanking is very bad idea...

I mean we can correlate blanking index and signals in result file. They should not point on the same area in data...

[Jason G]

Yes, I'm *attempting* Joe's 'full blank' now.  Don't know if I put it in the right place, We'll see.  I presume it should erase the 30 signals in JasonShort.dat without too much hassle. [If it appears to work I'll post SSE & SSE3 binary for playing with.]

[Later: Looks like 'Scorched earth' here, apart from best pulses all gone.]

Code:

Starting parse: .\testDatas\ref-astropulse_5.00_windows_intelx86.exe-JasonShort.
dat.res
Successfully parsed: .\testDatas\ref-astropulse_5.00_windows_intelx86.exe-JasonS
hort.dat.res
--- Found <ap_signal> tags: 40
Starting parse: .\testDatas\result-ap_5.00r69_SSE3_blankOverride.exe-JasonShort.
dat.res
Successfully parsed: .\testDatas\result-ap_5.00r69_SSE3_blankOverride.exe-JasonS
hort.dat.res
--- Found <ap_signal> tags: 10
Result      : Weakly similar or Different.

Unfortunately my hack doesn;t write indices.txt, but I do print a line to stderr whenever the randomize_indices() if block executes (stderr attached)

[Pending result from a full WU blank,I'll stop whining about the possibility of artefacts... and start whining about the blanking width instead  ]

[Leaps-from-Shadows]

Update from Cruiser:

First (and last) Astropulse work unit crunched with v4.37 optimized app completed on Main.  Task details.
Credits (759.31) divided by CPU seconds (112876.70) = 0.006726898 credits per CPU second.

Second Astropulse work unit crunched with v5.00 optimized app completed on Beta.  Task details.
Credits (868.56) divided by CPU seconds (89492.05) = 0.009705443 credits per CPU second.

I should have my first v5.00 unit crunched on Main in six hours, plus three more around this time tomorrow (two on Main, one on Beta).

[Haselgrove]

@All testers
Please keep eye on indices.txt in working directory (if it become non-zero length or not).
And if you keep result file, please, keep indices.txt too along with result and WU files.

ap_03no08af_B2_P1_00143_20081121_30048 (1065721003)
ap_03no08ag_B2_P1_00100_20081121_20392 (1066329522)

- both just started - have chunky indices.txt files (attached). If I'm reading the guesstimates right, these both have a high chance of pulsing out before their allotted span. Have suspended networking for a result capture.

That second one is a jewel. 2141 blanking indices, leaves about 6422528 samples to process so roughly 81% is blanked. A waste of CPU, but intellectually interesting. If it exits early I suspect the percent blanking reported in the result file won't agree with my calculations, it's done incrementally. In theory it should not be likely to exit early, in practice I'm not even willing to guess.

The indices.txt file isn't used for anything, just written during the initialization step which generates the internal array. It's basically the same information which was put into stderr.txt by the Linux 4.37 stock build, but BOINC will only report the last 64K of stderr. I'm glad they preserved it in this form even if it is ridiculously verbose.
                                                                       Joe

Both tasks ran full-length and have now been allowed to report. They both found a limited number of single pulses (4/7 respectively) plus 30 repetitive pulses. I have captured the output files and will upload the 'jewel' (only) to FTP. The other one can stay in cold storage until Jason's mum has got her bandwidth cap removed....

Note for anyone else assisting in this research: the indices.txt file is deleted from the slot directory when the WU finishes processing, whether networking is disabled or not - and of course, a WU can finish early at any moment. Since it's created at the beginning for the run and not changed thereafter, it would be best to copy it to a safe place as soon as the WU starts.

[Jason G]

  30  repetitive pulses in the ~81% blanked task? duh,  I'm just going to take a look and see what I can see... If full-blanked shows a similar pattern we (or the project, rather) has a problem IMO.

[Jason G]

Hmmm, arrchive doesn't seem to open.  Any problem during upload (besides speed ?)  LoL ... too quick on the trigger    I'll wait 'til the upload finishes eh 

[Haselgrove]

Upload complete - you can pull that trigger now.

[Jason G]

Got it, and looking at the result file.  All signals listed are same <time>, which doesn't make sense to me uury, so I'm looking for some periodic relationships between one repeating pulse and the next...

[Haselgrove]

Not quite all. The 'jewel' has seven pulses with different times, and the other one (not yet uploaded) has four - so I guess that the single pulses are accurately timed, and the repetitive pulses just have a dummy value inserted - which happens to match the <start> time in <data_desc>.

I'm slightly surprised by the blanking report at the start of the result file analysis: <percent_blanked> -18.750000 for the 'jewel', and <percent_blanked> -21.785156 for the other.

[Raistmer]

I'm slightly surprised by the blanking report at the start of the result file analysis: <percent_blanked> -18.750000 for the 'jewel', and <percent_blanked> -21.785156 for the other.

So, Joe's % estimate and Josh's differ... Interesting, how much % will be in our current totally-blanked run....

[Jason G]

Picked one of the non-repetitive signals at random to look at: [playing with numbers still...]

<ap_signal>
  <fft_num>30015488</fft_num>
  <peak_bin>30024704</peak_bin>

Which I (with less than novice skill) make to be be data_chunk_now: 7503872 + 9216 samples (A bit less than a third into the data chunk?) at sample 30024704

That would place that somewhere: in the vicinity of the blanking index:

"At sample 30035968, data_chunk_now 7508992, strength 12.390625 > 12.100000"

So the peak is 11264 samples before the index... On the edge of the blank perhaps shifted by dedispersion?  I'm not sure, but I think this shouldn't be there    [Now where did I see that with before?  20480...]

Oh $@!$#... here  I'll wait for Joe's expert analysis, but I reckon that one's an artefact [of some sort]

If at first you don't succeed, destroy all evidence that you tried.

- Murphy's Law