Issues with 840 Pros for CacheCade?

Has anyone been using Samsung 840 Pro SSDs for an SSD cache with LSI CacheCade and noticed any issues? I'm troubleshooting some very strange I/O on a new node using these SSDs.

this issue of "no capacitors" on any consumer-grade SSD's (except Intel 320), well Karl would disagree with you wholeheartedly that this isn't an issue, has been talked by all sides to death on this thread: http://www.webhostingtalk.com/showthread.php?t=1234660 so I ain't going to repeat myself again! I stand as a strong believer that you really should (or "shall" by my standard!) use Intel S3700's as caching drive(s) controlled by cachecade. main reason is that any cachecade-ready spin-drive based raid-10 equipped server would be easily $3000 or higher, therefore spending just $120 extra (100G S3700) or just $250 extra (200G S3700) to use S3700 is really a no-brainer! why take any chance on this critical piece on the data chain flowing from raid core to main array? let alone the 5 years of worry-free-of-read-only even you intend to cycle 1TB data on a small Intel 100G S3700 as caching drive every single day! as to use consumer grade SSD such as 830/840P/520's installed as array members on the main array, that's a very different scenario to consider. I'm kinda agree with Karl that you could choose to take chances with them in doing so. Karl is correct that we all have been using spin drives installed on disk array, with no "capacitors" either, since RAID was invented. if no-capacitor on array members was that "critical" to the "health" of disk array, regardless software based or hardware based, then we should have abandoned the "idea" of disk array all together long, long time ago. whether the ultra large buffer (512M on 840P!) can make it more vulnerable to have corrupted file system in sudden power loss event than spin drives with much smaller buffer (16M-128M), that's a debate for another day. using enterprise SSD's as member drives for main array are still very, very out of reach by most hosts due to the cost. even a moderate 4x 400G S3700 are FOUR GRANDS, drives alone! so, you can't hardly blame them making economic sense to use 830, 840P, 520...etc, etc. at least I won't!

Make sure your firmware of the RAID controller is fully up2date to the latest version. We also noticed extremely bad performance with the LSI 9266-8i on Intel 520 ssd's which were used for caching via Cachecade. It was working very well after we upgraded the firmware of the controller to the latest version.

OCZ Vector NOT Vertex. The Vector came after Vertex. Has good multithreaded results so will be good for hosting. I see I made a typo about Vertex earlier. Sorry.

That PDF refers to 840 and not 840 Pro?

same X9DRL-F board, same barebone CentOS 6.4, same dd write 0 (roughly 11GB write), 2x Intel 240GB 520 in mdadm RAID1 consistently gets 650MB-665MB/sec write speed regardless block size tested from 16K to 1024k! Intel 320 series and Samsung 830 series are basically in EOL, you just can't get them any more unless off eBay.

Are Samsung shipping the drives with this updated fw yet?

Are you getting any errors at all? Just to confirm as well, you are using a LSI direct product, not rebranded (which might have a custom rom)? What model are you using?

hmmmm.... which LSI card are you using? we have no issue whatsoever to do so on 9260/9260CV with the latest BIOS/firmware (v4.9 Jan-08-2013) installed. I personally had confirmed my saying just this morning, and my tech guys has been doing so for weeks since we discovered the dismal IO rate if we didn't enable the buffer on 840P.

I have. The 9266-4i with CV still does not seem to work well/ reliable on CentOS 6.4

OK thanks. I ordered a Supermicro E3 server for the office and will do some testing with various SSDs on a 9266 and a 9271, so I was just looking for some feedback

I don't know that there is a separate pdf for the 840, I will look, that being said do you really drop raid support on a "PRO" model and put it on the base? I don't think there is any question they can fix this with firmware, they just have taken the stance that they don't want to, and are pushing the sm843 drive. Hopefully enough pushback from user's will change their mind.

none of 1st tier distributors has even had SKU#'s listed for samsung SM843/843T series to this date. so, it would be a while before anyone could really buy them from retailers.

840 pros also don't seem to work on Adaptec 7805Q for max cache.

No errors, LSI 9271-8iCC (not rebranded).

It's a 9266-8i with the latest firmware (3.230.05-2100 Jan 08, 2013).

We've got a bunch of 9265's and 9266's running on CentOS6 just fine.

Be careful with 840 (pro) and sas2108/2208 controllers. LSI put these SSD's on the can't change Disk Cache list, not sure why, unstable dropping drives might be reason. Can someone please check what the setting is in Device manager (before any tweaks are applied with above) MSM won't allow the Disk Cache setting to be changed from 'No Change' This should mean if you change it in device manager the controller should not change it. There is a chance the controller may stop device manager from changing the setting.

"840 series" is the title at the top of the PDF... pretty broad, I doubt you can single out 'pro'.

I still feel like the 840s should be capable of better performance, even if they are cheating with the drive cache i cant see how the drive couldnt be coded in a way that even a raid controller works.. I am hoping we are just a firmware update away from success. I have tried to contact samsung and any talks havent made me feel as if they are working on it. That being said the actual firmware comes out of korea my guess any US based support doesnt know what they are doing. If i understand the issue correctly we think it has something to do with the fact that the drives rely on disk cache. but if it works in a single drive scenario and you never really outpace the cache to the point that a single drive's writes suddenly fall of the face of the earth (my raid array of 26 drives gets 5-25 MBps writes, and I could never get a single drive to perform that poorly) then you would think the cache operations if done correctly would be transparent to the raid controller as well.

With the old firmware or already tested with the latest firmware on Adaptec?

Be happy to chat on IM, but what settings do you have on the cache cade block set to?

will try to confirm this next time we have a chance to build one with 9266-8i or 9271-8i (they share the same BIOS/firmware). shouldn't be too long a wait in doing so....

Yeah, I have some too.. But seems the last few setups have not worked properly.. It's been kind of crazy... Maybe it has something to do with the x9scm-iif board, I think that is one difference here.

STATUS UPDATE: ENABLING DISK CACHE LEADS TO INSTABILITY Did some testing with the disk cache enabled. Results are at forums servethehome c0m/raid-controllers-host-bus-adapters/1653-should-sticky-samsung-840-840-pro-not-lsi-megaraid-compatible-2.html Enabling the disk cache makes SSDs drop from the array under load. Happened in two separate instances with two different Samsung 840 Pros with nearly the same sequence of events. Seems all we can do is be vocal about it to LSI.

You actually outlined my stance quite well, though one caveat. I DO feel that a larger/critical Cachecade type array is the perfect spot to use an SSD with a capacitor. It is for standard/general use where I do not see the need/concern.

I don't know how many times I'll have to repeat this, but: The Samsung 840 Pro issue has to do with high latency after a random write. This is evident in some extensive review tests and more reputable reviewers have already ruled out recommending this product for this reason. This is the actual problem. It could be a firmware / hardware bug, not sure. RAID just makes this issue worse as it's easily triggered under RAID. End of story.

Latest firmware installed to adaptec 7805q (latest avl saturday night), shows drives and can add to maxcache pool but they don't seem to get used. Havent checked ssd firmware, easier to just swap for intel 520.

The same as on other nodes running like normal :/ I've already ordered parts for another server - I'll configure it exactly the same and see what happens with some testing. Thanks for your time.

It depends on the architecture. Sandforce has no buffer/RAM at all by design to save cost and to stop this issue. Marvell and Samsung controllers for example do. It doesn't mean the buffer will be an issue. It depends on what it is used for and how it is implemented. Take a full note on the implementation and design please. Don't make a blanket statement without in depth on how the controller works.

I buy and test these on each generation, With the 830, my nVidia SATA2 platform (ASUS M2NPV-VM), would timeout and log errors in its SMART table during attempts to test with a simple read benchmark on HD Tune Pro. With the 840 Pro the drive will detect in the BIOS but communication is so unstable that the drive causes the system to hang while attempting to boot (as a secondary drive). I test on this platform and provide feedback to manufacturer support channels since most SSDs lock down to SATA1 on this platform (non-AHCI in this case, some later nVidia SATA were AHCI capable), but Samsung are consistently unreliable (830) or unusable (840). Until Samsung gets their compatibility testing up to a competent level, buying these drives is a crapshoot for anything but new systems. I would have said this terrible legacy compatibility was mostly an issue for the standard 840, but now that the regular 840 is using TLC any business environment should only seriously consider the 840 Pro (or higher capacity 840s at a minimum) Samsung needs to fix these compatibility problems on both product lines. They also need to fix the 830, but I don't have a lot of faith in Samsung supporting older product lines.

LSI used to support the 840 which has now dropped off their supported hardware list(unchecked), this happened shortly after LSI bought sandforce. Samsung uses their own controller now as well. Basically, LSI isn’t supporting what isn’t theirs.

Capacitor is certainly ideal if you use cachecade in read+write mode. For read only there's no chance of actual data loss.. just loss of cache temporarily until it's rebuilt. Although I wouldn't run cachecade in RW mode at all unless there's a Active/Active or Active/Standby SAN setup...

I do believe the issue is beyond a lone factor of "high latency after a random write". all the dd write tests mentioned in prior posts were "sequential" writes in nature, yet all we got were dismal results from these 840P in disk array. just about all benchmarks published by those reputable reviewers were done on single 840P as single drive configuration, and they did test it thoroughly with all sorta of random write tasks with varying queuing depths. if "high latency after a random write" was an issue, then they should have discovered these extremely low write-rates already! I would agree with cryptz that whatever the issue is, it would be more to do with the 512M write buffer on 840P. anyway, even with single SSD non-RAID configuration, a single Intel 520 SSD can have ~650MB/sec throughput all day long which is a good 40% faster than the ~460MB/sec you can get from single Samsung 840 Pro. thus, for single-SSD non-RAID server, Intel 520 is a preferred choice too, let alone 520 has (1) buffer-free, therefore worry-free of power loss (2) NAND's rated 5000 P/E cycles (3) 7% factory installed over-provisioning to make them even endurable. too bad, 480G 520 SSD's are in such extreme shortage....

it seems the new "05" Samsung f/w makes world of difference to LSI raid cards. so, may want to try the 05' for 840P installed on adaptec variants as well.

Have you been able to get the Samsung 840 Pro's to work on the LSI 9271-8iCC yet?

Seems the 9271/9266 doesn't allow you to change disk cache policy settings for 840P arrays. People in this thread have already reported being unable to change it with StorCLI and MegaRAID Storage Manager. I can confirm that WebBIOS doesn't provide the option, and attempting to execute "-LDSetProp -EnDskCache -L0 -a0" in MegaPCLI exits with the error message "Not Allowed to Change Disk Cache Policy." I'd venture to guess that the 9271/9266 is unable to recognize the existence of a disk cache on the 840Ps. We'll have to file support requests with LSI, and hope they fix it in the next firmware revision.

LSI's Interoperability Report for MegaRAID has been updated (possibly a new firmware coming on April 6th?), but still no sign of 840P compatibility. http://www.lsi.com/downloads/Public/..._List_SAS2.pdf In fact, they completely removed the column regarding CC2.0 support they had next to the SSD drives...

reviews of Samsung SM843 "enterprise SSD: http://www.storagereview.com/samsung...ise_ssd_review http://www.tweaktown.com/reviews/522...iew/index.html purportedly, 240G SM843 costs just $20 more than 256G 840 Pro. but it seems they are just about the same drive in many ways: 1. same Samsung MDX S4LN021X01-8030 SSD controller 2. same amount of 20mn MLC NAND chips (8x 32GB = 256GB) except that SM843 has 7% factory pre-set over-provisioning 3. same 512MB SDRAM buffer 4. neither SM843 has capacitors for buffer protection! what gives?! I suspect the added performance from SM843 simply is the result of 7% pre-set OP on it. bear in mind that you can also do user OP on 840 Pro to achieve the same thing. since SM843 is using the same MDX controller and 512M buffer, I strongly doubt it would do any better than 840 Pro with regard to the on-going compatibility issue with LSI hardwarw RAID cores. from these benchmarks, Intel DC S3700 shines brightly and convincingly.

Just an update, samsung said this which gives me some hope: Dear Customer, Thank You for contacting Samsung SSD support regarding your inquiries. The 840 Pro serias SSD are compatible and will work in RAID, the plain 840 SSD are not designed to be set up in RAID. Samsung tech support deos not support any RAID array. Thank You. So in a nutshell its supported but they will not provide tech support. My hope is that they will then at least make attempts to fix a firmware issue related to raid. I am trying to find the right person at samsung for adaptec to talk to.

IT has been DISCOVERED in reviews. If you read some later reviews on other SSDs they point this out. Read carefully. This is where I picked it up. Intel 520 will only be able to achieve the write speed you refer to if ALL the data is highly compressible. I'd test with random data and confirm. Most SSD can achieve the max SATA3 sequential these days. It's about the other aspects. Intel 520 is behind performance wise. A rather last generation product. Having said that, I was the one that recommended the Intel 520 to you

Updated 6x 840 Pros and LSI 9271-8iCC with latest firmware. Disk cache can now be enabled. Performance during the previously mentioned IOMeter test was stable at ~100,000 IOPS.

it's pretty much proven by a few users that Samsung 840 Pro as caching drive is not quite "compatible" with 9271-8iCC. we also have at least one client complaining about the same low or inconsistent array performance by using 840 pro as cachecade caching drive. so, at this point, we will start recommending Intel S3700 enterprise SSD as caching SSD. for low-budget user, Intel 520 SSD is a much cheaper alternative to S3700. of course, you can opt for the old Samsung 830 but it's pretty much discontinued by Samsung already, and it would be increasingly difficult to source one in a hurry. Intel s3700, 520 and Samsung 830 are officially on the HCL as Cachecade (CC) Pro approved by LSI.

Haaa...you've got me on this one big time! despite your tone, I MUST say a big "THANK YOU" to point this one out. I'm not afraid a bit to admit my own mistakes and ignorance, hopefully very occasional! after all, bringing facts and truth to benefit users and visitors to these forums is 10x more important than being shamed by my own ignorance and flaws. VERY, VERY true that any Sandforce based SSD drives just DON'T have buffer/cache on drive at all because that's the way how Sandforce controller is designed and functions. this includes all SSD drives from little-known names to big time Intel as long as sandforce is the chosen SSD controller. this critical fact does change things quite a bit! 1. since there is no buffer to be required on sandforce SSD's, therfore there is no need to have capacitors to protect buffer in the first place, and that's why not a single sandforce SSD on the market has capacitors installed in end products. 2. of course there are other factors to consider when choosing SSD for production server, but from pure power-loss-protection's point of view, Intel 520 series now in fact is "safer", therefore better "suitable", than Samsung 830/840/840P in production servers because there is no buffer to protect on Intel 520 or any other Intel SSD with sandforce. all data going thru sandforce are in 'pass-thru' mode with no data temperately stored on buffer to lose in power-loss events. it would be just like we purposely disable write buffer on HW raid card because no BBU is optional nor installed. 3. all benchmarks associated with sandforce based SSDs should be more "pure" than SSDs with Intel/Samsung/Marvell controller which do require write buffer (64M on intel 320, 128M on Crucial M4, 256M on Samsung 830, 512M on Samsung 840P, then 1GB on intel S3700). whether on-drive buffer can be truly "counted" or "inclusive" as part of true SSD performance in real world, that's a top for another day. at least benchmarks from sandforce are very raw, undisguised, and more "true-form" because there is no buffer/cache to influence those benchmark results. as we all learned by now, disabled/enabled drive-cache on 840P could mean the difference between 10% and 100% of the IO rate reported. I suspect some degree of bigger or smaller influence of buffer on-off would show on other SSD equipped with cache buffer on drive. 4. if enterprise-grade S3700 was not affordable to you, and by the same reason as (2), and despite Intel 520 has lower performance on average than Samsung 840P, Intel 520 should be "preferred" by hosts, who are very afraid of data-loss event induced by power loss, to use these consumer-grade SSD drives as either caching drive or array members on main array. no data on no buffer, no data to lose, pure and simple. the “issue” of reliability of sandforce itself as SSD controller and it's associated "bad" history of BSOD's should play how big a role in making SSD choice is another topic on another day. from the factor of buffer or no-buffer, therefore capacitor or no-capacitor, it seems this matter is settled now: among consumer-grade SSD's, Intel 520 is just a better choice than Samsung 830/840/840P on production servers, various higher or lower IOPS figures aside, of course. AND I will start to point out these pro/s and con's to my own clients from this point on. THANK YOU, concerto49!

We are having big problems with LSI 2208 and Samsung 840pro in Raid 10 arrays. Pathetic performance. Tried the latest drivers and even an unreleased version received from LSI and no luck. Anybody else having issues with LSI 2208 and Samsung 840pro in Raid 10?

Makes sense. I didn't realize LSI had bought SandForce until yesterday.

I don't know, but from everything I see it would appear the 840's flash is plain slow with writes, hidden by its massive cache. Works great for casual use but sustained writing will eventually cause issue once the buffer can't keep up. Just an uneducated opinion.

Could you cite these reviews? Because the only reviews I see of the 840 Pro's max latency show it's around average. http://www.storagereview.com/samsung_ssd_840_pro_review http://www.storagereview.com/images/...maxlatency.png http://www.storagereview.com/images/...maxlatency.png http://www.storagereview.com/images/...maxlatency.png http://www.storagereview.com/images/...maxlatency.png http://www.storagereview.com/images/...maxlatency.png http://www.storagereview.com/images/...maxlatency.png http://www.storagereview.com/images/...maxlatency.png http://www.storagereview.com/images/...maxlatency.png

thanks for the update ! Anyone using LSI 9260-8i with 840 Pros ?

Yeah, it's not happening with the 840 Pros at the moment. I found a source for the 830s I need (128GB), so hopefully they'll be in stock for a while.

Welcome. Thanks for the praises. 3. Sandforce controller uses compression. It's not exactly "pure" as it's faster on compressible data and dips on non-compressible data. This is reflected in benchmarks such as Crystal Disk and AS SSD where RAW non-compressible random IO is measured. However, real world speaking, compression helps a lot. Intel claims the 1GB buffer in S3700 is not used for actual data caching, but for internal house keeping. Whether that is accurate I'm not sure. I would say Sandforce's firmware issues should have largely been fixed by now, especially with Intel's custom firmware that's been heavily tested. It's a mature product. It's current problem isn't the firmware, but the speed as it is an aging controller. The newer Marvell, Samsung and Indilinx are a lot faster. Samsung 840 TLC: Even if you write 20GB/day, it's still more than 5 years on 128GB model. http://www.anandtech.com/show/6459/s...ce-of-tlc-nand

LSI-2208 is what's behind e.g. LSI-9265, so it's bound to have the same issues reported here.

Since you're saying that the SM843 is almost the same drive with the 840 Pro, then maybe it's not LSI, but Samsung that wants to force enterprise users to use the SM843. Probably something in the 840P firmware that "prevents" proper operation with CC2.0? Has anyone had a chance of testing the S3700 with CC2.0? I see the sequential writes for the smaller drives are relatively low (200MB/s for the 100G, 365MB/s for the 200G), so it's either a 4 x 200G RAID10 CC2.0 array or a 2 x 400G RAID1 CC2.0 array - since there's a 512G limit on the Cachecade VD. Then again CC2.0 only does RAID0 or RAID1...

more and more I like Intel 520 series as "general purpose" client SSD drives to be installed in production servers. unlike 840/840P, the performance out of Intel 520's is very true and raw, and it's NOT "enhanced" by some buffer RAM because it has none. two months ago, member HostSentry reporeted the sub-par performance from 840P running mdadm software RAID1 already. by comparison Intel 520's in mdadm ran much better in both sequential and random write (FIO benchmarks, comparing Intel 120G 520 to Samsung 256G 840P). http://www.webhostingtalk.com/showpo...3&postcount=23 so, with or without hardware RAID array, the performance level from Samsung 840P is really very questionable at best, looking back!

very interesting posts here. Since the Samsung 843(T) Models seem to be compatible to LSIs Raid controller (compatibility list from 5/3/2013) I have a short look at them. But they seem to be slower in 4kB write i/o then the 840 ?? 843T 15.000 842 11.500 or do I have wrong data?

What version is the firmware?

You can get 830's at low volume (especially 128's) on Ebay fairly easily. I snatched up a batch of 256's last week.

despite that there were some reported incidents of BSOD's from Intel 520's, but those seemed to be all associated with Windows' users. little to none were reported by linux users, at least I couldn't fine one from googling very hard. we do have a client asked us to install 50pcs 480G intel 520 on 25 dual Xeon E5 hex-core production servers (2 each running CentOS' mdadm RAID-1) under very heavy random IO situation in last 6 months. so far so good, and 0 BSOD-like incident. maybe we just need to start trusting Intel+sandforce combo more than we used to, especially for linux kits. as to write endurance discussion, the math is rather simple! 120G TLC has 1000 P/E max = 120000GB TBW (total byte written) limit. 120000GB divided by 20GB (w/perfect 0x write amp) a day means 60000 days = 16.4 years however, if 3x write amp (typical desktop/note non-enterprise use), then the endurance is reduced to 5.4 years. considering the heavy IO on production server, even just 5x write amp would reduce the endurance to 3.3 years. if 10x write amp or you need to cycle 100GB a day even with just 3x write amp applied, then a dismal 1.3-1.1 years would be the end result. so, it's totally up to user patterns and the implementation of GC/TRIM/OP to reduce write amp on production server. regardless, I will say 840 Pro MLC, if must be samsung, is still a much better, safer choice (3000 P/E) because it allows 3 times the TBW, let alone the better performance.

I know that. The issues reported here are mainly cachecade. I am asking about Raid 10 arrays made of Samsung 840pro and not cachecade.

I had pretty much the same questions. Would be nice to hear peoples performance results with the S3700. We are currently using 2 x Mushkin DX's with 550MB/s write/read pretty much however its sandforce based which we heard is despised against when used for a cache purpose. On the plus side, WebBIOS allows us to enable disk cache for them though.

Chong - the Intel 335s seem pretty solid so far as well.

reviews show them as being slower on writes, that being said i am not aware of any retail location both physical and online actually selling the sm843 yet, atleast i cannot find any.

Always a pleasure. Samsung 840 Pros: DXM05B0Q LSI 9271-8iCC: 23.12.0.0013 Samsung has posted DOS ISOs here. http://www.samsung.com/global/busine...downloads.html I'm not sure if this would work, but there's a PDFwDownload command in MegaCLI for updating drive firmware through the controller. Convention is Code: MegaCli -PdFwDownload [offline][ForceActivate] {[-SataBridge] -PhysDrv[0:1]}|{-EncdevId[devId]} -f <filename> -aN|-a0,1,2|-aALL and command documentation is on pg. 249 of the MegaRAID SAS Software User Guide: http://www.lsi.com/downloads/Public/..._SAS_SW_UG.pdf You could try it with the 1 MB .enc file in the zipped file on Samsung's website: http://www.samsung.com/global/busine...o_DXM05B0Q.zip I'm not making any guarantees it'll work though, as Samsung explicitly states, "Zipped files must be used with Samsung Magician Software." It's likely just a statement that means other flash methods are not supported. If anyone is willing to try this method, please report whether the flash was successful. It's painful to manually connect each 840 Pro in the array to an onboard SATA port. Moreover, Samsung Magician Software forces a restart after a single drive update.

How about just using the 9271-8iCC for the FastPath. I don't need the cachecade, since it's going to be an all SSD solution. So the issues only appear if you are using the cachecade feature?

Ok. More information. The BSOD is a not only Sandforce's fault. I watched over this whole event when OCZ first rolled out Sandforce 2 drives. It has to do with driver implementation on the Windows side as well as other factors. SATA drivers have been updated over time to help combat this and so Sandforce have also introduced workarounds for it. Sandforce has had other issues though, not Intel related. E.g. they rolled out a firmware update that disabled TRIM and didn't enable it back for months. Intel had no issues as it used custom firmware. AS to write amplification - it isn't something that you change. The write amplification is based on the controller. Sure there is base and worse case scenario but it differs per controller and how it handles writes. This is why Samsung was able to use TLC in Samsung 840 - because the controller reduced write amplification. It is for the same reason NANDs are able to go through die shrinks. The controller has improved to reduce write amplification. P/E cycles were at one stage 5000. It means nothing. Make then write amplification was a lot higher. As said, please look at the whole package, not just NANDs and controllers separately.

the thread title may mention "cachecade", but last 2 pages of posts were all about compatibility issues between LSI hardware RAID cards and Samsung 840 Pro's, with or without cachecade. let me sum it up! all LSI 2208 (dual-core ROC) based RAID cards 9265/66/71 series just can't enable the on-drive 512M buffer on which Samsung 840P wholly depends to generate performance. the result is that even 4x 840P RAID-10 is having slower performance than single SATA drive! even LSI 2108 (single-core ROC) based RAID cards such 9260 series is having troubles with 840P drives! although RAID BIOS on 9260 card can enable write buffer in 840P, but it still would have very slow write if block size was smaller than 64K. thus, basically you WANT to avoid the combo of any LSI hardware RAID card and Samsung 840P drives at this time! the only exception is 3ware 9750 series which is also based on LSI 2108 ROC but it has totally different BIOS/firmware than LSI 9260 series, and Samsung 840P RAID-10 on 9750 card does fly regardless write block size.

I wanted to throw in my experience. I have 24 840 pros (256gig model) in a server. Originally I had a dell h710 and an expander and I thought that was causing problems. I just switched to an adaptec 72405 and direct sata connections to each drive. If I put all 24 drives in raid 0 I get about 2000MBps read and write. if I put the same drives in a raid 50 I get about 1300 read and only 25 write. the adaptec has an hba mode. if I try the drives in hba mode the OS basically locks up completely and spams the event log with: The IO operation at logical block address XXXXXX for Disk X was retried. followed by : The system failed to flush data to the transaction log. Corruption may occur in VolumeId: E:, DeviceName: \Device\HarddiskVolume8. (A device which does not exist was specified.) I honestly think the problems you guys are seeing are less LSI and more SAMSUNG. I have had basically the exact same experience with both vendors. I know the adaptec allows me to turn on the drives cache (it wants to by default) The drives are very unstable in hba mode. In raid mode the controller doesn't seem to be complaining about drives going offline but I suspect something is causing the horrible write performance. I have the array wb and read cache disabled, I suspect enabling it would mask the problem a bit but I am trying to identify the root cause. I opened a ticket with adaptec 2 days ago, no response. I called Samsung and they are supposedly forwarding my info up the chain but they said they do not officially support raid arrays. I hope the fact that the adaptec 72405 falls flat on its face in hba mode (and technically isn't raid, the magician software does see the drives) helps get some result.

Those test results could be even better for the 520s if he was using a 180G model or larger, as there's quite a difference in advertised random 4k read/writes between the Intel 520 120G and the 180G. Intel 520 120G Sequential Read 550 MB/s Sequential Write 500 MB/s Random Read (8GB Span) 25000 IOPS Random Write (8GB Span) 40000 IOPS Intel 520 180G Sequential Read 550 MB/s Sequential Write 520 MB/s Random Read (8GB Span) 50000 IOPS Random Write (8GB Span) 60000 IOPS

ok I ve just found a review on thessdreview.com comparing it to Micron P400m and Intel s3700 In Germany the non T is already available at caseking.de

Would be good if someone can test if it works with megacli. Having to pull the drive 1 by 1 and updating on a Windows dedicated server that is "spare" will be a major issue.

Just wanted to update you guys. I just installed the 256GB 840 Pro model on a 9271-8iCC and it seems to work like the 830s... So perhaps the issue isn't just plain incompatibility between the RAID cards and the 840 Pro line. Guess we need more tests with more models.

To be clear, is the buffer still an issue even when combined with a BBU or CacheVault?

I've made multiple support requests asking about enabling disk cache policy on the Samsung 840 Pros. Each time I make one I get no answer, and the request closes itself automatically. They're avoiding the question entirely. At least with other hardware brands I had hope they would fix compatibility issues, but with LSI I'm not holding my breath. Buyer beware.

Yeap, that's definitely not right... I'm also leaning towards the assumption that Samsung wants the 840P out of the data center market, in order to push sales of their SM843 "enterprise" SSD.

Aandtech reported that surprisingly 335 series (20nm NAND) allows pretty low P/E cycles: http://www.anandtech.com/show/6462/i...on-the-ssd-335 regardless it's 1000, 1500 or 3000, it's far lower than 5000 P/E cycle rated for 520 series. on top of higher endurance and performance, the per-GB price of 520's is just 10%-20% higher than 335's, then I rather recommend 520 SSD's to clients. although 480GB version of 520's is pretty tough to source currently due of extreme shortage, but the supply situation will eventually improve. 335 series is limited to 240GB max.

the sm843t is factory overprovision and has drive activity light function enabled. these 840pro's are sensitive to unrecognized commands. They tend to freak out and lock up. IBM and DELL like to send commands to their chassis and/or port 0/1 that cause them to freeze up. Lack of OP is unacceptable! 25% OP or pick another SSD for 840 pro. The 9266/71 in hp chassis without a battery work flawlessly with esxi and heavy load tuning to esxi 5.1 advanced settings. We see in the Advanced Performance Disk Statistics peak of 3 million KB/s with two controllers servicing 4 samsung 840 PRO 512gb each. You must check out megascu and make changes to the controller ini files (mobilenvidia servethehome are good places to find the changes) otherwise the combination will not function well. Good combinations: 10-25% OP RAID-10 H700/512 (fastpath is free!) NO BATTERY Dell 12th generation controller, suggest using all slots other than 0/1 Good Combination: 0% OP - RAID-10 FASTPATH 9260-8i or 3rd revision M5014 - dl360 G60 8sff sas-expander with dual target. One controller is only option due to sas expander. SOLID esx 4. Heavy disk i/o tweaks in place. I regret not giving 10% OP Great Combination: DL380 G7 - dual 9266-8i (no battery! notice the theme here!) - 4 samsung 840 pro each. Of course you could just buy a P420/1gb FBWC and use hp saap 2.0 key to enable disk read cache 750gb per 1gb FBWC - 1.5TB per 2gb FBWC - the remaining FBWC will be used to write cache the drives. HP Smartcache is only cachecade 1 - read only - but works in all G6/G7/Gen8 systems with adequate cooling. I've got a whole stack of these for our SE1220 Lefthand P4300 G2 (aka dl180 G6 with intel nic's and ilo2) - converting them to ssd accelerated VSA's. 1. Overprovision - or else. 2. Don't fit to ibm/dell slot 0 or 1 - and definitely NEVER put a raid-10 span on slot 0/1 (ask me how i know!) 3. If possible use HP servers. DL180 G6 with dual quad cores go for like $400 and can hold a $150 emulex 10gbe dual nic and a P420/1gb FBWC with 2-4 ssd in the rear and 12 LFF in the front. DL380's are selling for $355-550. Can expand to 16 SFF, dual controllers (get the dual risers!). DL160 G6 $200-400 8SFF, DL320 G6 $200-400 8SFF. All of these have sufficient airflow for a P420 or 9260/9266. Sorta. #1 problem for these controllers is heat. The 9260/9266/9271 and especially the P420. They cannot run at 80C. The P420 idles in a dl180 g6 at 56C and the LSI 9266 at 66c. This is too hot. The SE1220 servers have double fans rigged up. This drops the temp even more but the LSI controllers do not shut down when they exceed 84c like the HP P420. They start to flake out. You must take action to ensure extra cooling, remove the thermal pad and use better heatsink/arctic silver . Like 10gbase-T nic's the heatsinks on these things are way too small and simple poor wiring choices(hp dell ) lead to insufficient linear airflow and ducting. Anyone with a brain can enhance cooling. Do the math. 840 pro is stupid fast. These cards have puny heatsink. Folks do not think when placing them about this and you have a perfect storm of pushing the limits of the adapter and slightly buggy firmware that can be avoided. The bugs in the LSI controllers are ginourmous. The battery is the #1 problem. It is very easy to correlate the battery failure and cycles to samsung 840 failure under stress. This doesn't happen with the P420 (PMC Adaptec SRv8-6gbps). Adaptec Maxcache 7805/71605 are the same controller without the nic HP ACU/ORCA. HP has two new low end solutions for ssd only folks- the dl360e has bare option Intel Sata 6 ports (optional 512 FBWC , skip it) and LSI2308 riser with 8 sas - the FBWC is the same. If you take the riser out and put the non-lsi 2308 riser in, the fbwc and intel sata 6 port enables. Can't have both at the same time. The intel can run in normal mode -non raid- and so can the LSI2308 - it is called H222 IT mode. Likewise the gen8 has HP Virtual Smart Array for esxi,redhat, and windows. Giving you just that. The target market in marketing is quite clear "EMERGING USE: IE SSD". They know it is the fastest path to have software raid - and they gave it to you guys. DL360e gen8 are cheap with either option. SSD Caching is meh. It's ok. Write-back caching through consumer drives is insane. It is more brutal than just ssd-raid. Let your application (oracle/sql) do the work. Or just setup SSD and use it properly. Several RAID-1+0 with horizontal and vertical partitioning to protect against cage failure will save you. If you can afford to use separate raid-1 volumes, you reduce write amplification over RAID-10/1E. Avoid Raid-5/6 - seriously unless you are rocking enterprise drives just avoid it folks. You must understand LSI firmware is buggy as heck. You must tune the controller using tools you shouldn't have (MEGASCU) to tweak features on all megaraid controllers to achieve your stability. If you are using OEM megaraid controller, be mindful of the out of band functions that hose these drives. Good luck - the samsung 840 pro with the right chassis and controller is solid. Rock solid. Use Raid-10. Avoid spans on slot 0/1 and consider read-only caching with 750-1.5TB using the HP P420/1gb fbwc. (I'm telling you a really good secret and i'm guessing the folks who are catching my hint will get a price that will amaze you, and that is a direct ship from tech data no less)

If there are people who want to try using MegaCLI that don't know much about the command options, I can walk you through them. Code: [offline] This option makes the targeted physical drive (PD) go offline prior to flashing it. Code: [forceActivate] This option forces the new firmware to activate without a reboot. Code: {[-SataBridge] -PhysDrv[EnclosureId:SlotNumber]}|{-EncdevId[devId]} This part of the command specificies which device to flash. Code: EncdevId[devId] This option would flash enclosure firmware. Code: PhysDrv[EnclosureId:SlotNumber] This option would flash a PD, with enclosure given by EnclosureID and slot number in that enclosure given by SlotNumber. If you wanted to flash a PD directly connected to the controller, the EnclosureId would be 252 and the slot number would be the controller slot number. Code: [-SataBridge] This option would flash the SATA bridge firmware of a SATA bridge connecting a PD with address given by PhysDrv. Code: -f <filename> This part of the command specifies the filename of the file to flash. Code: -aN|-a0,1,2|-aALL This part of the command specifies which controller (adapter) is connected to the targeted PD. When there's only one controller present, just use a0. So if someone wanted to flash an 840 Pro connected to the first port on their controller, they could run Code: MegaCli -PDFwDownload -PhysDrv[252:0] -f DXM05B0Q.enc -a0 assuming that they only had one controller and that DXM05B0Q.enc was located in the same directory as MegaCLI. For people who want to be sure about the identities of the PDs they are flashing, they can run Code: MegaCli -PDList -a0 to get a list of information for all PDs, which will include enclosure ID, slot number, device world-wide name (WWN), and SCSI INQUIRY data (Vendor ID, Product ID, and Product Revision Level). To get this info for a particular PD, run Code: MegaCli -PDInfo -PhysDrv[EnclosureId:SlotNumber] -a0 If you have a lot of 840/840 Pros or want to flash for many systems, you could script this to run the update only on drives with old firmware. Though it would likely be easier to use StorCLI instead of MegaCLI for this as it was made for scripting. The StorCLI command for flashing is Code: storcli /cx/ex/sx download src=<filepath> so the analagous StorCLI command for the MegaCLI example given above would be Code: storcli /c0/e252/s0 download src="DXM05B0Q.enc" To get information about all drives on a given controller, use Code: storcli /cx show all You could then parse that to get Enclosure ID and slot numbers for all PDs, and run Code: storcli /cx/ex/sx show all to determine which drives are 840 Pros that require flashing. More information about StorCLI and command syntax/semantics is available in Chapter 5 and Appendix C of the MegaRAID SAS Software User Guide.

Just to confirm you did use it for CacheCade, right? Did you test it from a VPS? By the way, working like the 830s as in same step? 840 Pro should be a LOT faster.

sure, SSD controllers can have varying success in minimizing write amp, but factory-installed OP (over-provisioning; ~7% for consumer SSD to 32% on S3700) and user OP can drastically increase write endurance which has the same effect as minimizing write amp. 20% user OP could increase endurance level (or pre-set TBW limit in becoming "read-only") 2-3 folds. if installed on RAID card which has no support for any GC/TRIM stuffs, user OP is pretty much the only thing you can do to raise endurance level.

I am already trying the new Adaptec 7 series.

No. The real issue is along the lines of Samsung 840 Pro introduces very high latency after a random write. We've dropped them on all builds after this. (smile) Might be fixed in a firmware, might not.

Recap There seem to be multiple, distinct problems with the 840(P)/LSI combo and the 840(P) in general. Lets codify what we've found out thus far: Without drive caching, the 840(P) has poor random write performance in RAID 0. I've confirmed this myself, and others have as well. I found that 4k32QD random writes take a 90% hit when drive cache is disabled. cwl@apaqdigital had similar results [1] . With drive caching, the 840(P) is unstable with LSI 2208 controllers [9265/9266/9271] during heavy reads. Specifically, I found the 840(P) to be unstable with the 9271-8iCC. And this controller, like the other LSI 2208 controllers, did not allow for the enabling of disk cache without tweaks. It may be safe to assume that LSI disabled this option in the LSI 2208 controllers because they were aware of the issue. The instability results in drives dropping out of the array, but they can be re-inserted after reboot. With drive caching, the 840(P) performs poorly with the 9260 at low block sizes (<64k). This was found to be the case by cwl@apaqdigital [2] . It is unknown at this time whether this phenomenon carries over to other controllers. With drive caching, the 840(P) performs poorly in configurations other than RAID 0. This has been verified with multiple controllers, specifically by cryptz on an Adaptec 72405 in RAID 50 [3] , by Andreas on an LSI 9207-8i in RAID 5 [4] , and by ZB1 on an LSI 9286-8e in RAID 6 [5] . The main issue in all cases is 4k32QD random writes. Only the Adaptec has been confirmed to have had drive cache enabled, and I will request the others to give feedback. supermacro on an LSI 9265-8i in RAID 5 [6] also found poor 4k32QD random write performance, but this is likely due to a disabled drive cache. The only known counterexample to this is by mkrad on an LSI 9266-8i in RAID 10 [7] , which got 4k32QD random writes at ~300MB/s. It is possible that he may have had controller cache set to write-back, as doing so has been shown to boost write performance by cryptz [8] . Analysis Problem 1 doesn't require much explanation. Disk buffer gives faster writes, although the extent of this effect depends on the SSD. In the 840(P)'s case, it matters quite a lot. Problem 2 is more complex. I ran the IOMeter test mentioned in my previous post [9] 3 times with the disk cache enabled on my 9271-8iCC. The test consisted of 8KB transfers with 80% read, 80% random at 64QD. In each of the 3 runs, a different 840P dropped out of the array after about 3-4 minutes. After every test, the disk was re-inserted into the array without any apparent signs of data corruption. Each of the three runs had similar results. The read and write IOPS started out high, then quickly dropped and appeared to stabilize. However, after a minute or two these stabilized values began to plummet at increasing rates. When read/write IOPS reach around 60,000/10,000, a disk drops out of the array, and test errors begin increasing rapidly until BSOD. Selecting the right display parameters for IOMeter reveals some information. In particular, max read/write latencies go as high as 42 ms for both read/write under load, which at first glance is large in comparison to other drives [10] [11] . However, the test conditions in these reviews are likely oriented for consumer use. Enterprise testing reveals that these max latencies are fairly normal [12] . What's interesting is that the random transfer speeds at which a disk drops out, which equate to 480MB/s read and 80MB/s write, are approximately the same as those found at 32QD in CrystalDiskMark when the disk cache was disabled. It suggests that the problem may lie with the way that the disk cache controller handles requests when the disk cache is nearly full. I posted the LSI controller event logs in another forum [13] for the first run. The other runs were very similar in the sequence of events. I was able to decipher the meaning of these error codes from a SCSI commands reference . What's happening is that the controller is trying to read a 64k block from the PD, and the controller times out on the read (Event #1). The controller resets the PD (Event #2), and tries the read again only to fail with another timeout (Event #1). Another reset occurs (Event #2) and the controller attempts another read, but the PD errors saying it's not ready (Event #3, this is just the controller sending a command too early after reset). Another read happens, but it keeps timing out in the same fashion (Events #1, #2, #3). After another attempt at a read and a timeout (Event #1), the PD resets (Event #2) and attempts the read one last time, only to get an unrecovered read error from the PD (Event #4). The controller changes the state of the PD to SHIELD (Event #5), resets (Event #2), and attempts to run a background short self-test, but the PD errors saying it's not ready (Event #6, this is again just the controller sending the command too early). The command is sent again, and the diagnostic fails, after which the device resets (Event #2), the event is recorded (Event #7), and the controller changes the PD state to FAILED (Event #8). The gist of it is that the reads from the 840 Pros under enough load eventually timeout, and these timeouts repeat, culminating with an unrecovered read error that drops the disk from the array. The natural response to this would be to say that LSI 2208 controllers may be experiencing a timeout because the timeout period simply isn't long enough to handle the max read latency of the 840P under heavy load. The problem with this explanation is that enterprise testing [14] reveals that the maximum read/write latencies under load easily exceed 42 ms for many SSDs. Another problem is that it doesn't explain the plummeting read/write IOPS during the IOMeter test. That is to say, if these tests were a true reflection of RAID 0 performance of the 840P over time with six drives, then many more people would be complaining. One might be quick to blame lack of TRIM for this performance drop, but with an IOMeter test file of 1GB and 1TB free on the drive, it's very unlikely. Tests with an LSI 9260, which is reportedly stable with drive cache enabled, will confirm whether this drop in performance persists. Another possibility is that the protocol that governs how LSI 2208 controllers communicate with the 840P is to blame. Perhaps the Caching Parameters for the SELECT_MODE SCSI command are inefficient for the 840P, or perhaps the read is requested in such a way as to result in an uncorrectable read error. It would take a much deeper understanding of the controller's and SSD's architectures to be able to discern this, much deeper than either LSI or Samsung are willing to elaborate. Problem 3 is a little perplexing. Normally, smaller block sizes would offer better random read/write performance. If it's something exclusive to the 9260, it may again be a protocol issue in the controller. Problems 4 could be due to inconsistent random write performance of the 840(P). With RAID 1, RAID 5, and RAID 6, random writes occur on every drive when a block is written, compared to RAID 0, where random writes are evenly distributed amongst the drive. Furthermore, the write is not considered completed in RAID 1, RAID 5, and RAID 6 until all writes to PDs are completed. Probabilistically speaking, with many drives this latency is going to increase, especially if there's large variability. Though there's also a possibility this could be due to other weaknesses in the drive. Conclusions Enabling disk cache seems to be the solution for people experiencing slow 4kQD32 random write speeds in RAID 0, if possible. For those experiencing terrible performance with RAID levels other than RAID 0, enabling write-back cache on the controller may be an option if there's a BBU.

samsung released 05 firmware, i am in the process of updating my drives to test (it takes a stupid amount of time to update 26 drives 1 at a time with a server shutdown after each one). the ISO cannot detect drives through the adaptec series 7 controllers so i am stuck doing 1 at a time in windows.

Should explain things a little more

can you guys, who are still suffering from the issues of inconsistent or constantly-low IO rate from Samsung 840P installed as caching drives under cachecade pro, double check whether drive-buffer (512M) on SSD itself is turned on? by default, LSI BIOS turns the SSD's buffer off, and we found that "off" setting made 840P performing dismally slow (10% or worse) comparing to "on" setting. our bench tests were not on a server with cachecade pro running, and we don't currently have a server with cachecade installed, therefore we can't do this experiment ourselves. maybe you can shine some lights whether it makes any difference between buffer on and buffer off?!

BBU/CacheVault will only protect the RAID cards write cache, not the drive cache.

Call them after you open the ticket and it sits for more than a day. For me it's worked to escalate the tickets and I've gotten callbacks from engineers...

The only thing that thing is good for is random read/write and being stable at it. The sequentials are terrible. It won't win you any praises in benchmarks by you know who :p

UPDATE KamiCrazy from STH reports [1] that he's seen a forum post where someone relays information from Samsung about why 840(P) performance is so terrible without disk cache. It would appear the disk cache is additionally used for large parts of garbage collection and wear-leveling, which calls into question the endurance of this drive when the disk cache is off.

I would encourage people to try the new firmware. It is a little earlier to tell but I am noticing much better raid 5 write performance in windows and have not had a drive drop yet.

A new firmware update has been released for LSISAS2208 controllers (23.16.0.0012). At this time I am unable to check whether this update substantially affects performance, although from the readme it would seem it does in some cases.

added... a soft-boot may be required to re-initialize the LSI BIOS once drive-buffer setting is turn on or off. also, this drive-buffer setting is nothing to do with the embedded 1GB or 512M buffer on LSI RAID card itself.

Has anyone had any luck improving the performance of their 840 pros? I have an LSI 9286-8e card with fastpath and 8x840 pro 512GB SSDs. I initially set this up in a RAID 0 and got fantastic speeds; high sequentials and good random IOPS . This is a production server though and I needed some form of redundancy (plus over 2TB of space) so configured a RAID 6. Sequential speeds seemed about right but random IOPS were terrible (4k32QD writes = about 8000). I know the write performance is worse with a RAID 6 but I wasn't expecting it to be that bad. Random reads should probably be higher too. I've been in contact with LSI but they're taking their time to get back to me. Just e-mailed Samsung too. LSI settings - Always write back, no read ahead. Like other posters, the drive cache option is greyed out and stuck on 'unchanged' but when I configure one of the settings it does say in the log that the disk cache policy is enabled. Just seen that LSI have released a firmware update a couple of days ago. I've not flashed it yet as this server is supposed to go live today! has anyone else? And if you have, have you noticed any improvements? There's nothing in the release notes to suggest the issue might have been fixed.

What is a good alternative SSD you guys are using with the LSI CacheCade controllers?

I have an Intel RS2BL080 (LSI 9260-8i), I'm going to migrate the array over to that controller during the weekend and run the same IOMeter tests I did with the LSI 9271-8iCC. If the drives fail again when disk cache is enabled during the test, it's a sign that the 840/840Ps can't handle heavy reads. Which would indicate this is Samsung's doing, as the SM843 is specifically marketed for being able to handle read-intensive workloads. This is beginning to remind me of the fiasco with WD over TLER.

This kind of goes along with my theory, these disks may be fundamentally unsuited to sustained writing especially with heavy queues...

You're running RAID5 on SSDs?

I'm having similar issue with the OCZ Vectex 4 F/W 1.5 on 9271-8i. Under full load, the drive seems to fail. OCZ has firmware 1.5.1 now. Has anyone tested it? LSI F/W Package: 23.9.0-0018. Anyone test the new 23.12.0.0013 F/W just released?

I'd be happy to give you any needed feedback as my CacheCade node has not been put into production yet, but I don't see how I can enable/disable each SSD's drive buffer from MSM. Only thing I can enable/disable is the Disk Cache Policy setting on the spinning disks RAID-10 array (to which the 4x Samsung 840 Pro's CacheCade VD is attached), but I guess that's not what you mean.

Right, just installed the firmware. Had a ten minute window and didn't need to restart anyway. The new FW hasn't made a difference. Anyone have any ideas?

Intel DC S3700

Except the fact that even the "Enterprise" drives still don't work with the LSI controllers... so if Samsung doesn't fix this, it will just mean going with another vendor -- which sucks... for us and for Samsung, but for Samsung more so.

More mdraid issues... http://www.webhostingtalk.com/showthread.php?t=1257454

im just testing it but yes, my long term goal is to run zfs, but even then i would be running the equivalent of raid 50. Other then raid 10 i am not sure what you would expect, no one is going to be using 0 unless they are caching.

I am considering doing RAID10 on 6-8 of these Samsung 840/840 pro drives using Dell PERC 6/i and H700/H710 RAID controller. Does anybody have any experience with this setup?

On storcli the physical drive cache setting is "pdcache" -- set to on. On ours it won't let us turn it off for SSD VD's though. ie. storcli /c0/v0 set pdcache=on Side note, just use the S3700's in the future. Only a bit more $/GB and totally worth it.

Have you tried RAID 10?

If you're looking for budget drives, you can find people selling OEM PM830's in fair quantities (20-50 at a time) with only a few hours on them. Last time I looked, it's the OEM PM830's that are on the compatibility list, not the retail 830's...

I wanted to post a quick update. I am seeing dramatic read improvements by enabling wb cache on the array. I can understand the write improvement, can anyone offer a guess as to why reads are doubling? read cache is not enabled. this is a 24 840 raid 50, 6 legs: same exact array without wb cache: Sequential Read : 1273.857 MB/s Sequential Write : 331.129 MB/s Random Read 512KB : 746.859 MB/s Random Write 512KB : 159.690 MB/s Random Read 4KB (QD=1) : 16.765 MB/s [ 4093.0 IOPS] Random Write 4KB (QD=1) : 9.941 MB/s [ 2427.0 IOPS] Random Read 4KB (QD=32) : 301.432 MB/s [ 73591.8 IOPS] Random Write 4KB (QD=32) : 119.006 MB/s [ 29054.2 IOPS] with wb cache: Sequential Read : 3408.090 MB/s Sequential Write : 3519.181 MB/s Random Read 512KB : 2713.099 MB/s Random Write 512KB : 2713.221 MB/s Random Read 4KB (QD=1) : 72.745 MB/s [ 17760.0 IOPS] Random Write 4KB (QD=1) : 71.135 MB/s [ 17367.0 IOPS] Random Read 4KB (QD=32) : 428.195 MB/s [104539.8 IOPS] Random Write 4KB (QD=32) : 368.675 MB/s [ 90008.5 IOPS]

Anyone have any experience with the vectors or intel 520 in a large raid array non 0, 1, or 10? Looking to switch at this point

RAID5 with SSDs is a really bad idea. RAID5 by it's nature (lots of parity rewrite) has lots of write cycles.

That's great news. Has anyone tried the ISOs with mdraid?

are you using Samsung 840P? if yes, are you having any IO rate issue even with pdcache on? if no, what do you have? just maybe that's where the culprit is: the drive buffer on 840P, only the 840P series, got stuck at "disable" mode by LSI firmware so that constant-low IO rate or inconsistent IO rate as the result. wish LSI would look into this one....

I'd like to use raid 10 but can't, need at least 2.5TB of space.

Intel 520's are a good alternative we've used them without issue.

Without WB, writes will completely block reads. So WB will always affect read performance to more or less degree.

Can't say that I'm impressed by the Intel 520's in RAID10 (on an LSI 9266-8i)... max speed I could squeeze out of those on a regular dd test was ~500MB/s. I'm going to try the Vectors next week and will let you know.

the environment is an iscsi target for my house, the write load is insignificant.

Nevermind.

+1 The DC S3700 is an awesome drive.

no luck here either! we also tried the latest 5.6 firmware (Mar-02-13) from LSI on 9266/9271 cards (dual-core LSI 2208 ROC), but we still can't change "disk cache" on Samsung 840 Pro from "no change" to "enabled" like 9260 cards (single-core LSI 2108 ROC) can. we also found that these LSI 2208 based RAID card can't change (drive buffer) setting on Intel S3700 enterprise SSD either. it always stays at "no change". so, this issue is not exclusive to Samsung 840/840P.

Plextor M5 Extreme Pro or OCZ Vertex are the top performers right now.

I would expect a benchmark focusing on reads or writes to not have such a significant impact though, I do not believe there are any writes to speak of during the read test. The system has 0 user access at the moment. That being said I didnt want to derail the thread. Basically my initial point was that I didnt feel this was an LSI only issue. The adaptec card I have does not work well at all with these drives in HBA mode. In raid 0 they work rather well. If i turn off the drives onboard cache (not the volumes)via the raid card options the raid 0 performance dives down significantly. going from 2Gps Read, and write to 1300 MBps read and 20 MBps write. The array cache is off in both of these examples. If i create a raid5 array the peformance is the same as the raid 0 with the drive cache disabled, leading me to believe that somehow in a raid 5, 50, etc, the drives cache is being turned off even though its set to on. In the case of the raid 5 and 50 you seem to be able to cover up this issue by enabling the wb cache for the volume itself (not the individual drives). This seems to resolve the issue but this will not really be a long term solution once the server is in production and the raid array cache is constantly being saturated. Or I should say I am not willing to let it get put into production with this issue. I could get much better write performance with spinning drives.

was that 4 intel 520s or more? how are they in a raid5?

It greatly kills performance too - RAID5 that is on SSD. Not recommended.

Has anyone flashed an 840 through an LSI RAID card yet?

I too have been finding poor performance with the Samsung 840 Pro SSDs on the LSI MegaRAID 9271-8iCC (not necessarily just with CacheCade). I'd like to try the suggestions mentioned in this thread. How do I turn on the SSD Disk Cache? The LSI Documentation only describes how to view the SSD Disk Cache Policy Setting in Controller Properties, not how to change the setting. Must I clear my current configuration, and set the policy when I create a new array? Or is there a way to change the setting without clearing the configuration and losing the data on the array?

What do you advice will work? (ignore Intel 520 and Sandforce) Does this mean anything with a cache/buffer fails, i.e. Samsung 830, OCZ Vector, etc etc?

Well shoot. Will have to return the box of Samsung SSD's then if it wont work with CacheCade. Two storage servers with LSI 9271-8ICC, bummer. Samsung Pro's look extremely promising to use, to bad LSI messed it up

Unless you have the array mounted RO there's no way to ensure there's no writes. Even small ones could mess up your results especially with 840P's which are slow as dog doo with their internal cache off. Have you tried the same test with spin disks or other SSD's?

4 x Intel 520 180G in RAID10

any new insights cryptz? im greatly interested, cause im willing to buy a cachecade ssd, either 830 or 840pro

Please bear in mind that Samsung has a dedicated range of Enterprise SSDs validated to work with LSI and give high performance in the data center. Check out the Samsung SM843T SSD.

from LSI RAID BIOS screen, look into the "logical view' (the right window pan), identify the "virtual device '0', xxxx.GB, optimal", then just click on it mouse pointer (or navigate ny key strokes), then the property screen of the chosen VD (virtual device) should show up, identify "disk cache" under "policy" section, and change it from "disabled" to "enabled". no! you don't need to redo the whole array nor to redefine virtual device again. however, a soft-boot to re-start the RAID firmware may be required (actually I'm not even sure whether that is required?).

what about LSI 9260 series?

Samsung 840 Pro have issues beyond CacheCade anyway compared to some of the others on the market right now.

Can anyone tell me what they get or have seen for read performance results of 4-8 drive setups in Raid-10 on any LSI 92xx card?

any chance you could try them in a raid5? i am curious as to how they perform, i am currently leaning towards them over vectors.

If you can still find Samsung 830's then go for it, they're proven to work 100% with LSI's CacheCade.

Which is very hard/impossible to find though.

I'd be wary of enabling cache on a non-capacitor backed SSD, especially on a RAID array... N drives more chances of corruption in power loss... I think the verdict is in, 840Pro is a no-go for RAID...

in terms of setting "disk cache" option, there is no issue with LSI 9260 series. 3ware 9750 series is an alternative as well since 3ware BIOS always enables disk cache, and there is no option to disable it AFAIK.

Has anyone tried Kingston SSDs, particularly the V200 series?

forums.servethehome.com/hard-drives-solid-state-drives/1610-anyone-4-x-samsung-840-pros-raid5-lsi-card.html#post13774

I can confirm that 840 pro's are NOT good for mdadm software RAID either! we tried 2x 256GB 840P installed on supermicro X9DRL-iF (2x SATA-III ports by Intel C602 south bridge in AHCI mode), configured mdadm RAID1 (64-bit CentOS 6.4 barebone), fully sync'ed all RAID-1 partitions, then ran the basic dd write 0 tests with varying block sizes (from 16k to 1024k), and the all we got were 20MB~22MB/sec write. by comparison, 2x Seagate barracuda 7200.14 SATA-III in adadm RAID-1 can do 200MB+/sec easily. we even tried disconnecting one 840P, then ran the same dd tests on remaining 840P in "degraded" RAID1 mode, and still got the same result. next, we re-installed CentOS 6.4 on single 840P without adadm, then all write 0 tests became very zippy and achieved 450MB-470MB/sec write speed consistently! this proved that Samsung 840P is only good for single drive configuration without any disk array regardless software RAID or hardware RAID. at this point, the only sensible thing to do is to put 840 Pro SSD's on the black list of not-to-use for disk array of any kind!

Has anyone had a change to play with the OCZ Vectors in production?

SM843T SSD parts have recently become available to wider customer market in Europe including UK. Please feel free to contact me if you would like the name of a local distributor. FYI I work in Samsung Memory in Korea.

That would be the spinning drives' disk cache then, not the SSDs'. The CacheCade VD has no option other than the Write Policy (Write Back or Write Through).

an update! it seems 3ware 9750 series does a lot better with Samsung 840 pro SSD's than all LSI 9260/9265/9266/9271 cards. current issues from the combo of Samsung 840 Pro and LSI raid cards: 9265/66/71 (2208 dual-core ROC) - can't enable on-SSD 512M buffer which causes very low write rate because performance from Samsung 840P series heavily depends on on-drive buffer. 9260 - low write rate when block size is smaller than 32K we just tested some 840P SSD's on 3ware 9750-4i card (drive-buffer always in "enabled" mode; no user option) this morning, and found the write rate was consistently higher than 700MB/sec on every dd write-0 test we've done, block size ranging from 16K, 32K, 64K, 256K, 512K, 1M. some tests even showed 1.1GB-1.3GB/sec! can't say the same thing about LSI cards which gave out some write rates as low as low 100M/sec which was even slower than single SATA drive. ironically, LSI owns 3ware! so, this must be some sort of firmware conflict(s) between LSI and samsung 840P with regard to on-drive buffer.

STATUS UPDATE: A SOLUTION HAS BEEN FOUND. Seems another forum has a thread detailing this same issue. forums [D@WT] servethehome [D@WT] c0m/raid-controllers-host-bus-adapters/1653-should-sticky-samsung-840-840-pro-not-lsi-megaraid-compatible.html Users report that LSI 9260 and Intel RS25DB080 (LSI 9265) allow the disk cache policy of Samsung 840 Pros to be enabled. Problem seems confined to 9266/9271. Someone named "mobilenvidia" presented a solution to disabling the disk cache policy lock, though it uses an older utility called MegaSCU. Google MegaSCU for a download link. 1) Run MegaSCU -adpsettings -write -f mfc.ini -a0 2) Edit the mfc.ini file, setting "blockSSDWriteCacheChange" to 0 instead of 1. 3) Run MegaSCU -adpsettings -read -f mfc.ini -a0 4) Reboot 5) Disk cache policy can now be changed (through MegaCLI, MegaPCLI, or StorCLI) I ran CrystalDiskMark after enabling disk cache policy, and 4k write speeds were MUCH better. HOWEVER. I ran IOMeter with the following settings: Transfer Size Request: 8 KB Read: 80% Random: 80% Outstanding I/Os: 64 Number of Workers: 24 Maximum Disk Size: 1 GB Run Time: 10 MIN HBA was an LSI 9271-8iCC with one VD [6 256 GB Samsung 840 Pros in RAID 0]. IOPS were around 90,000+ initially. Dropped with time, but that was expected due to no TRIM. About 4 minutes into the test, HBA starts beeping. Shortly after was BSOD. After reboot, go into WebBIOS, one of the drives was missing. It may have just been a bad SSD, I hadn't used them all that much until then. Though we can't discount that the workaround may have bricked the drive. Until more people are willing to test this and determine whether this occurrence was a fluke, USE AT YOUR OWN DISCRETION. TL;DR - Solutions exists, but may be unstable.

here is the official response from adaptec: We haven't tested these SSDs officially, but we've seen already several customers of us having 840 SSDs fail or with performance issues. We have contacted Samsung and they told us, that 840 and 840 Pro SSDs are not designed for use in RAID environment. 843 and 843T should be used instead. Simple RAID-arrays, e.g., RAID-1 or RAID-0, shouldn't be an issue, but with everything else you run into danger of instable arrays by disappearing or failing devices. The 25MB/s do not look strong indeed, but it's not a weakness of the controller. It just waits until the drives (SSDs) are ready to proceed with the next data write operations. Consider that parity information on all 24 SSDs has to be distributed and written. That takes some time.

So disappointing...

finally, some happy news to report! after applying the latest "05" firmware (05B0Q) released a few days ago, the 2x Samsung 128GB 840 pro we configured as RAID-1 device on LSI 9271-4i card did perform 1.0GB-1.1GB/sec sequential write speed consistently with block size varying from 16K to 1024K when we did the dd write 0 tests. each dd test was roughly 11GB in size. with the old "03" firmware, they would score just at a dismal 5%-10% level. the latest LSI v5.6 firmware on 9271 card still can't enable/disable the write buffer on Samsung 840P, but it seems 'always enabled" becomes the default setting resulted from the new Samsung 05 firmware instead of "always disabled" and that certainly makes all the difference in the world for 840P drives. good news to report for mdadm software RAID as well! after we removed the same two 128GB 840P from LSI 9271 hardware RAID1, and installed mdadm software RAID-1 (CentOS 6.4 64-bit) on them, and we've got the same 1.0GB-1.1GB/sec write speed from the same dd write 0 tests (bs=16k ~ 1024k). about a month ago, the result was 20MB-22MB/sec from the 2x 256GB 840 pro in mdadm RAID-1 with the old "03" firmware. the 05 firmware update process is a mixed bag of good's and bad's. you must use Samsung's "magician" software for windows to do the update and there is no DOS nor ISO version of firmware updater. you couldn't update the Samsung firmware if it was installed on RAID card nor it was connected to USB port! you must connect these SSD drives to on-board SATA ports so that "magician" software could "see" the 840P drives. the good new is that once 840P's were detected by magician software in windows, the new 05 firmware update would be automatically notified and done by clicking one single button if internet connection was present where the "magician" software was running.

Hi all I have installed the LSI FW 3.270.65-2578 (17 June 2013) and the 840 Pro FW 5B0Q. My Configuration: 1 x SSD for CacheCade 2.0 Pro (Wrrite Back enabled) and 4 2TB Disk in Raid 5. I know Raid 5 is not the Best Performance Solution. But if i disable CacheCade oder set it to "Read only", my write Performance is stable at about 200MB/sec. (ISO / MKV copy Job in Windows Server) But i enable the "write back" mode, the the transfer goes down after a few seconds to 2-3 MB/sec. for 15 - 20 seconds, the up to 100MB/sec. for a few seconds and the again to 2-3MB/sec. What's the Problem? Have others here in the forum the same issues? Thanks a lot, Mike

well, we just did a few servers with 4x 256G 840P RAID-10 on LSI 9260CV-4i, and the LSI BIOS did allow enable-disable drive cache when 840P's are attached as array members to the main array. purported by a few of you guys, it appears that LSI BIOS can't allow enabling-disabling drive cache on any 840P only when it's attached as caching drive for cachecade. so, my best bet: this IS the culprit of low IO rate when 840P is serving as caching drive because LSI BIOS automatically disables the 512M buffer on 840P, and no mechanism provided to users to enable it. are you listening, LSI boys and girls?

Any way to get LSI to revamp the firmware on the 9266-4i for 840Ps?

Have you tried the OCZ Vertex 4 with CacheCade?

Any idea how this cachecade thing work for SSD? let's say I have a 10bay HDD, 2bay being ssd, set as raid 1 and the rest being normal 2TB sata hdd And if the ssd is being use as cache, for the cachecade pro thingy, there will be an option for me to select as SSD cache pool and the rest of the 2TB all as raid 10, and cachepro will be automatically do the rest?

thanks for the confirmation ! Looks like Intel 320/520 series is best for raid arrays then or the old Samsung 830s too ?

Thanks for the update.. shame about firmware update method.. web hosts using 840pro will have a fun time updating the firmware !

Samsung 840 Pro is now formally supported by LSI for MegaRAID cards.

WTF, how do you get 840 Pro no go for RAID from this conversation? The ONLY thing being discussed here is CACHECADE on LSI cards. That is it. So, verdict is LSI needs to fix their gear.

mine usually die at rejecting I/O to offline device this seems to be an LSI 9266-4i / CentOS 6 issue and not CacheCade though.

The Vertex 4 has shocking read performance. Wouldn't use it.

that's too bad! I was hoping Adaptec could become the savior of these Samsung 840P... is anyone running these 840P's on 3ware 9750 series? would love to see some benchmarks and user input about whether the 3ware array could stay stable under high-load or not? SM843 does not look very promising (low-endurance "regular" MLC, no capacitor, same MDX controller) at all! but, SM843T (up to 960GB) looks pretty nice! it's based on high-endurance eMLC, equipped with capacitor for power-loss protection, enhanced firmware, and it costs just 25% more than SM843 purportedly: http://thessdreview.com/daily-news/l...ny-213-update/

Anybody tried the Samsung SM843? They are on LSI Compatibility list and performance should be the same as 840pro.

Thanks Chong, that's good news indeed. In for results of long term.

Our Samsung 840 Pro seems to work fine with our LSI-9266-8i Do we need to do a firmware upgrade?

Yes, I can also confirm that when I create a RAID10 array out of the 4 x Samsung 840 Pro's, the Disk Cache Policy setting is grayed out.

I've notified LSI tech support a few times about the "compatibility" issues with Samsung 840 pro, yet the latest v5.6 firmware just released in Mar/2013 for 2208 dual-core ROC based RAID cards didn't improve things at all. so, good luck with LSI. Samsung's response was even more comical: "840 Pro SSD is consumer grade product! please use Samsung enterprise class SSD drives for enterprise servers", something in that effect. too bad, Samsung 830's were so smooth and trouble-free.

So were you referring to the Vertex 3?

I just got a good laugh, I checked the Samsung website to see if it said anything about the 840 not being supported in raid, it doesn't. even better this document says raid is supported http://www.samsung.com/us/system/con...et_FIN.pdf.pdf

I know some people have said that they've tried the 830's are those recommended or are they in the same boat as the 840/840 pro? I am also gathering that the 840/840 pro is not advisable in software or hardware raid. I ask because it looks like performance varies with software raid (good or bad).

our internal tests with the new 05 firmware were pretty limited in rather small scale. I would encourage that any host, who owns large# of 840/840P drives in production now, wants to do his own f/w updatings and testings in small scale first before spending time and resources to update f/w on all of them. it wouldn't be "fun" project at all to pull all 840P's from servers with disk array then put them one at a time on a windows workstation just to do the f/w update, then put them back on the disk array.

Is anybody using the 840 Pro with Cachecade Pro 2.0?