Confused with SDFileSystem.bas

[liak]

Dear all, especially the author of SDFileSystem, Steven,
I am exploring the above module for nearly a week. Still I am confused with it. I have tried from learning some important basics and SPI protocols of SDCard, thinking that it may help my understanding. But the more I read, the more confused I am! I also went through some basics of FAT16 structure. Same outcome.
I agree that, I don't have to know too detail of the SDCard SPI protocol as the headaches have been taken by Steven. But the module offered too many functions...
The Problems
I am actually trying to do something which I think is very simple. I hope Steven or anyone can offer some suggestions. My project requires me to generate and store a large amount of data. Previously I stored all this in EEPROM, but now I have exceeded the largest capacity of a single EEPROM, so logically, I move to SDCard.
For my project, I don't require any directory or complex file structure; or a single file will do. I thought of writing the SDCard just like writing an EEPROM. Can I write directly onto the SDCard like writing to EEPROM byte by byte (or sector by sector)? And can I read in the reverse manner?
Please help.

[Raistlin]

You can write to SD via SPI without formatting , but you must write your data in 512byte chunks (sectors) so you will need to buffer your data before each write operation , so you have enough data to fill the sector.

Not sure, but I would think that there are low level commands in the module (not been playing with SF for long). I'm 100% steve can point you in the right direction though.

[TimB]

How big was your largest Eeprom.

I use one that's 16meg bytes

[Steven]

The SDFileSystem module contains two private subroutines, WriteSector & ReadSector, that write and read the 512 byte buffer. These are kept private, as the module is intended to write files in FAT16/32 format and not raw data to the SD card. However, you could adapt the module by making these public and filling the buffer yourself. You would then have to keep track of the sector number yourself and increment each time. You could also use the Init command in the module to initialise the card.
Having said all this, it may be easier to write the data to a file and read from it, even if you don't need this facility - you would then not have to adapt the module at all. The only cost would be a higher code overhead and slightly slower execution. If you go this route, feel free to post any code if you've had a go and I'll help you get it working - what you want to do should be quite easy, so long as your write speeds are not very high.

[liak]

Dear TimB and Steven,
Thanks for the swift reply.
As for Tim, I am using 25LC1024, which is about 130Kbytes. Never thought that, there is anything larger than that! Spare my ignorance. I thought that's the largest in production. May I know what EEPROM that has 16Meg bytes which you refered to? I think this monster will cost me a lot! Anyway, most people still think it is easier to go direct to SDCard. Don't you agree?

And Steven, yes. Thanks a lot for your reply. Yeah, I noticed the two subroutines which you made private. I was trying making it public before posting this thread.
So, you agree that I can practically throw away the files and directories if I want to, right?
I am not as fluent in SF as I want to (will read some more!), especially at very low level (asm). I am having difficulty in understanding these parts:

Code: Select all

Public Sub WriteSector(pSector As LongWord)
'not the entire subroutine.........
#if SD_REINIT_SPI = TRUE
   ReInitSPIPort()
#endif

Repeat
      CS = 0                             // Send Cmd 24
      If SeekResponse($FF) = $FF Then    // Wait for ready
         If SendCmd(CMD24, pSector) = $00 Then
            SendByte($FE)                // Write start token 
            Index = 0
            FSR0 = @Shared.CurrentSectorBuffer
            Repeat                       // Write data block
#if SD_SPI = SW                          // Software SPI version - use SendByte

Do we need to re-initialize the SDCard SPI everytime before using it and under what situation? If yes then why is this set as an option? Can I do away with it to cut down overhead?

As with this line:
FSR0 = @Shared.CurrentSectorBuffer
I will have to declare the structure TShared and variable Shared as public in my program in order to access it directly right? Sorry but please spare my ignorance again, I can't find the definition and reference for FSR0 and reserved symbol @ as well, making me a bit uncomfortable to proceed, can explain a bit on this?

On the SDCard sector structure. The sector 0 is reserved for MBR, am I right? So we should never start count with sector 0, correct?

Thanks a bunch. I will post my codes once I make the changes for you to inspect.

[Steven]

OK, going through your points:

You can do away with the SPI re-initialise code - that is designed if you have an application with more than one device on the SPI bus. If the SPI bus has to be re-configured for that device, using the #option in the SD module will ensure that it automatically re-configures it for the SD card each time it is accessed. If you have nothing else on the bus, you can either remove this code, or turn the #option off.

Yes, you will have to make TShared public.

FSR0 is a PIC register used with POSTINC0, allowing indexed addressing of RAM. You set FSR0 to the address of the first byte to read and then each POSTINC0 instruction reads a byte and increments the address for the next read. You can do the same thing for writing. This is a helpful technique for writing to a series of sequential RAM locations (e.g. for arrays). The symbol @ is used to obtain the RAM address of a variable, here the start of the CurrentSectorBuffer.

As for the MBR, yes, Sector 0 contains this. If you don't need to use the card later for PC/camera use etc, you do not need to protect this. If you will need it later, overwriting this (and for that matter, any of the early sectors of the card) will destroy the formatting - Windows might not even recognise it for reformatting if the MBR and possibly some other early sectors are missing. It might be safer to start writing at e.g. Sector 100 (but this is only a guess). Even if you do corrupt the format, you could still write a small PIC progam to re-perform the full format - this should format the card from any state, even if you have overwritten the MBR.

You might also want to find the card size so that you don't write past the end of the card. If you want to do this for an unformatted card, you should look at DiskSize in the Format command.

[liak]

Dear Steven and all,
Hi! Sorry for late reply. Been away for few days; not able to test the reply given. Thanks again Steven for such exact and simple answers. I am quite clear now about how to proceed. But some more problems I encounter on the way.
I understand that the basic unit of the disk is a sector (512bytes), buiding up then we have n sectors in a cluster (depending on the disk size) and from the clusters forming up the entire disk. So the entire disk will be

DiskSize = 2^16 * n * 512 bytes Right?

The problem in my understanding is now:
We can only access a location address of 2^16 in FAT16, but the disk definitely has more sectors than this. In fact, correct me if I am wrong, the FAT16 address is pointing to the cluster address rather than sector address. If so, I am lost in using the ReadSector and WriteSector. The pSector variable is supposed to point to the sector number. Can you show me how to access the "lost sectors"? Or am I missing something?

[liak]

Dear Steven,
I have read the SD module more carefully, seeing the functions and subroutines for FATSectorToCluster and ClusterToFATSector. Are these for the calculation of the pointer location to access my "Missing Sectors"? I don't have a clear idea on the algorithm in these subroutines. Sorry to trouble you for explanation. Thanks

[Steven]

You are right that each sector is 512 bytes and that a number of sectors make up a cluster when using FAT16 or FAT32. You are also right that the number of sectors in each cluster depends on the disk size - the larger the disk, the more sectors there are in a cluster.

The disks themselves are accessed sector by sector and you have to specify the sector address when reading/writing using the SPI bus. If you are not using FAT16/32, you can ignore the concept of a cluster entirely.

When using FAT16, the FAT table is used to keep a track of the clusters used in a file. The file is written as (assuming 4 sectors per cluster), e.g.:

Cluster 1 - Sector 1, Cluster 1 - Sector 2, Cluster 1 - Sector 3, Cluster 1 - Sector 4, Cluster 2 - Sector 1, Cluster 2 - Sector 2, Cluster 2 - Sector 3, etc...

It is down to the program code to keep a track of which cluster it is in and which sector of each cluster. The FAT table is a list of clusters used - not strictly the address, but the cluster numbers, starting at cluster 2.

The FATSectorToCluster and ClusterToFATSector functions are used to find the cluster referred to by an entry in the FAT table (at a certain position in a certain sector of the FAT) and vice versa.

As for the disk size - you cannot obtain it in quite the way that you describe, as FAT16 allows for upto 2^16 clusters, but in practice not all of these possible cluster addresses will be used - there are likely to be slightly fewer clusters on any given disk. If you look at the Format command, it shows how to obtain the true size of the card in sectors using the SD Command 9 to read the cards CSD register. Note that this method will also include the sectors used by the FAT tables, the root directory and the boot record.

I wonder if you are trying to make this more complicated than you need to? I'd advise that you either:

(i) choose not to use FAT16/32, in which case you can ignore clusters entirely and use an adapted part of the format command to find the disk size and ReadSector and WriteSector etc, or
(ii) use FAT16/32 and use the library without alteration.

If program size is not a real issue and you are not looking for the very fastest speed possible, I'd do the latter...

Regards,

Steve

[rubenpena]

Liak:
If you do not need PC compatibility, you may use flash memory:
1._ Data Flash from Atmel , comes with 2 and 4 Megabites in 8 pin soic IC.
and cost $ 1.57 at Digikey (AT45DB161D) (2 MB.)
2._ Data flash from ST- They have up to 16 MB. (2,4,8 and 16).
Atmel also have a 8 Mb. but the case is more difficult to mount.
All that memories have SPI interface and works very fast.
You can use 4 chips with different chip selects to fill your needs.
I hope this helps...
Greetings...
Ruben de la Pena V.

[normnet]

For a ST 32 MB flash example see M25P32 PIC Audio
Warning other compiler.

Norm

[liak]

Dear Steven,
Thanks for enlightening me. I finally see that I was lost as I promptly put "Confused with SDCardSystemFile..". Yeah. I was wrongly thinking in the line of FAT when in fact there is no need of any FAT at all.
Still a couple of things I will need to straighten later.
Will get back to you later. Regards.

Dear Normnet and rubenpena,
Thanks for the info. Will keep these for future reference. But I think SDCard is the format that suites my application more currently. Thanks again.

Liak

[liak]

Dear Steven and all,

I am starting to get used to the SDCard now. After correcting my understanding. Thanks to you again.
I have made the necessary changes as following:
1. Making Shared as public array buffer
2. ReadSector(pSector as longword) and WriteSector(pSector as longword) both as public subroutines

I am able to access the SDCard now, both writing and reading. The next problem that I am tackling now is to keep the sector count correct. After browsing through the Format() subroutine in your codes, I figured that the maximum number of sectors on disk would be given by this:

Code: Select all

'********************************************************************************
'* Name    : DiskSizeKB (Public)                                                *
'* Purpose : Find total number of clusters on disk                              *
'********************************************************************************   
Public Function SectorCount() As LongWord
   SectorCount = (Disk.LastCluster - 1) * Disk.SectorsPerCluster
End Function

Is this correct? Because I am not using any LCD output, I am having difficulty in confirming the correct value for this. Is the formula correct?
Please confirm. Thank you. And I am using 1GB SDCard.

I am counting this:
SectorCount = 1024^3 bytes / 512 bytes per sector
= 2,097,152 sectors
Correct?

Regards.

[liak]

Dear all,
Before I proceed further, can anyone tell me how fast or any idea that PICs can read a typical SD card?

Thanks.

[Steven]

For the card size, you will have to access the cards CSD register - since you are not using FAT, you do not really want to be using anything that refers to clusters. This way, you can also slim down the Init command to remove the FAT section that reads and checks the FAT format.

As for speed, with the full FAT code, running at 40MHz (10MHz with x4 PLL), you can get average write speeds of around 60KB/sec and average read speeds of around 90KB/sec (assuming hardware SPI is used).