Android - Scoped storage OS 11

Every Android updated version has something for developers and users alike. The Android 10 version has many user-friendly and performance enhancement features. The Android 10 recently improved the file access level to protecting the application and user data. It has been increased security level that is good, but it also increased the work of developers to handle storage.

Storage Structure before Android 10

Before Android 10, an application could access any file on the device using the File APIs if user has granted the storage permission. The storage was divided into shared storage and private storage.

• Private Storage: All Android applications have their own private directory in internal storage Android/data/{package name}. The private storage’s was not visible to other apps.
• Shared Storage: Apart from private storage was called shared storage. The shared storage includes all the media and non-media files being stored. Any app could access this part of storage if it have storage permission.

 

problems With old storage structure

• If i want to upload or update any media file into the system, which means access to a single media file. So why am I asking for the whole storage access from the user? 
• All your files like, medical prescriptions or bank documents, being accessed by all the installed apps. If you have provided storage permission to any installed application.
• If i'm uninstalling the application that has performed file downloading action. The downloaded file will not delete and it'll increase the storage size that'll show the issue of insufficient storage.
  

Now, Google has introduced new storage form that called Scoped Storage in Android 10. An app would be provided access to the storage blocks which has relevant data for the app.

What is Scoped Storage?

• The Scoped storage is new storage structure of Android OS that prevents apps from having unrestricted access to the filesystem on the device. Application sand-boxing is a core part of Android’s design, isolating apps from each other.
• The Android OS will bind storage to owner apps so that it becomes easier for the system to locate relevant files of the app. It is useful when the app is uninstalled because all the data related to the app is also uninstalled. All the data related to the app will delete. It'll manage the device storage space.
• Now, an app can't access all files on the device if the user has granted the storage permission because internal app directories and external app directories are private. So, all downloaded files like photos, private documents, videos, and other potentially sensitive files are not to be used by another app.
  
• Media collection contributed by other apps can be accessed using READ_STORAGE_PERMISSION permission. WRITE_STORAGE_PERMISSION permission will be deprecated from the next version and if used, will work the same as READ_STORAGE_PERMISSION.
   

The new storage structure in Android look like below:
 

Now private storage is the same as before, but shared storage is further divided into media and download (non-media files) collection.

Thing Keep in mind if going implement scope storage

• If you going to target your application API 29 without making any changes and accessing files by using the standard File APIs. You’ll find those API's are no longer work. Instead, you have to rely on the Storage Access Framework in order to open files or folders. If you are displaying media items from the MediaStore. You have to work exclusively with Uris, not file paths (also, in order to delete or overwrite media items you have to ask the user for permission to modify those specific files, simply requesting the WRITE_EXTERNAL_STORAGE permission no longer works).
• Is Scoped storage required for all apps on Android 10?:  It is only required when your targetSdkVersion is set to 29. So the legacy apps continue to work. Also, even if you are targeting API 29 you can still use the legacy storage by setting android:requestLegacyExternalStorage=”true” on the application tag inside of AndroidManifest.xml. However, you should implement Scoped Storage in your app as soon as possible because Google says it will be required for all apps that of targeting Android 11. So, if your app uses the legacy storage model and previously targeted Android 10 or lower, you might be storing data in a directory that your app cannot access when the scoped storage model is enabled. Before you target Android 11, migrate data to a directory that’s compatible with scoped storage. 
  
• How to access simple files?
  In order to access files on the device, you can use the Storage Access Framework (SAF). By using ACTION_OPEN_DOCUMENT a dialog will be shown to the user where the needed documents can be selected. There’s also ACTION_OPEN_DOCUMENT_TREE to ask the user to select a directory and ACTION_CREATE_DOCUMENT in order to save files.
  

 

Batch operations

The Scoped Storage didn’t allow batch operations that modify media items like overwriting or deleting a group of photos. Before Android 10, you were limited to either showing a permission dialog for each media item that is not very user-friendly if you are working with many files at once, or you going to convert the media Uri to a SAF (Storage Access Framework) Uri and work with the file using SAF operations.

• Batch deleting: Let say, you going to delete two images. Now, in Android 11, you can request permission for the two images at the same time by calling MediaStore.createDeleteRequest and passing the Uris of the MediaStore items you want to delete:
  

  private fun deleteImages(uris: List<Uri>) {
    val pendingIntent = MediaStore.createDeleteRequest(contentResolver, uris.filter {
      checkUriPermission(it, Binder.getCallingPid(), Binder.getCallingUid(), Intent.FLAG_GRANT_WRITE_URI_PERMISSION) != PackageManager.PERMISSION_GRANTED
    })
    startIntentSenderForResult(pendingIntent.intentSender, REQ_CODE, null, 0, 0, 0)
  }
  
  override fun onActivityResult(requestCode: Int, resultCode: Int, data: Intent?) {
    super.onActivityResult(requestCode, resultCode, data)
    if (requestCode == REQ_CODE && resultCode == Activity.RESULT_OK) {
      // Image deleted successfully
    }
  }
  

  As you can see above code, we checked permission first before going to write to any of these Uris. If you did not check permission before going to perform any action. It'll show permission prompts, including those for which you have already holding write access. The above code produces a single permission prompt for all images. There are also two other methods that work similarly, MediaStore.createFavoriteRequestand MediaStore.createTrashRequest, in order to mark a media item as favorite and to mark it as trash, respectively.
  

• Batch overwriting: Before going to perform batch overwriting, we have to request for the permission as shown above. We need to call MediaStore.createWriteRequestinstead.

  private fun overwriteImages(uris: List<Uri>) {
    val pendingIntent = MediaStore.createWriteRequest(contentResolver, uris)
    startIntentSenderForResult(pendingIntent.intentSender, REQ_CODE, null, 0, 0, 0)
  }

  In this case, the permissions are tied to the lifecycle of the Activity and in this case also we can’t request a persistable permission. That's why, if we need to perform some long-running operations with the data that available outside of the Activity, we need to place Uri into Intent’s data field or into a ClipData, like so:

   override fun onActivityResult(requestCode: Int, resultCode: Int, data: Intent?) {
     super.onActivityResult(requestCode, resultCode, data)
     if (requestCode == REQ_CODE && resultCode == Activity.RESULT_OK) {
       val intent = Intent(this, MyService::class.java)
       intent.data = uri
       intent.addFlags(Intent.FLAG_GRANT_WRITE_URI_PERMISSION)
       startService(intent)
      }
   }

• New Additions: In Android 11, we have a new addition that is called “All Files Access”. It is designed for applications that need broad file access, like file managers. Now, we have to declare the MANAGE_EXTERNAL_STORAGE permission and direct users to a system settings screen where users will enable the option to allow access to manage all files for the app. Which allows applications to read and write to all files within shared storage and access the contents of MediaStore.Files.
  

  Also, now applications that have the READ_EXTERNAL_STORAGE permission can read a device’s media files using direct file paths and native libraries, which helps avoid problems while using third-party libraries.
  

How to access simple files?

In order to access files on the device, you can use the Storage Access Framework. By using ACTION_OPEN_DOCUMENT a dialog will be shown to the user where the needed documents can be selected. There’s also ACTION_OPEN_DOCUMENT_TREE to ask the user to select a directory and ACTION_CREATE_DOCUMENT in order to save files. 

 

Storage Operations

Let see some basic storage operation that frequently implements in the application:

• Select a file: You can use ACTION_OPEN_DOCUMENT flag to open the file picker app that allow the user to choose the file. To show only the types of files that your app supports, you can specify a MIME type.
  

  // Request code for selecting a PDF document.
  const val PICK_PDF_FILE = 2
  
  fun openFile(pickerInitialUri: uri) {
      val intent = Intent(Intent.ACTION_OPEN_DOCUMENT).apply {
          addCategory(Intent.CATEGORY_OPENABLE)
          type = "application/pdf"
  
          // Optionally, specify a URI for the file that should appear in the
          // system file picker when it loads.
          putExtra(DocumentsContract.EXTRA_INITIAL_URI, pickerInitialUri)
      }
  
      startActivityForResult(intent, PICK_PDF_FILE)
  }

   
• Select a folder: You can use ACTION_OPEN_DOCUMENT_TREE flag to open a folder.
  

  fun openDirectory(pickerInitialUri: Uri) {
      // Choose a directory using the system's file picker.
      val intent = Intent(Intent.ACTION_OPEN_DOCUMENT_TREE).apply {
          // Provide read access to files and sub-directories in the user-selected
          // directory.
          flags = Intent.FLAG_GRANT_READ_URI_PERMISSION
  
          // Optionally, specify a URI for the directory that should be opened in
          // the system file picker when it loads.
          putExtra(DocumentsContract.EXTRA_INITIAL_URI, pickerInitialUri)
      }
  
      startActivityForResult(intent, your-request-code)
  }

  The above access will be valid till the user reboots the device. If your app wants to persist with access, while accessing the URI using the content resolver, the content resolver has to call the take persistable Uri permission method. If you iterate through a large number of files within the directory that’s accessed using ACTION_OPEN_DOCUMENT_TREE, your app's performance might be reduced.

• Create a file: You can use ACTION_CREATE_DOCUMENT flag to save a file in a specific location. It'll not overwrite an existing file. If your app tries to save a file with the same name, the system appends a number in parentheses at the end of the file name.
  

  // Request code for creating a PDF document.
  const val CREATE_FILE = 1
  
  private fun createFile(pickerInitialUri: Uri) {
      val intent = Intent(Intent.ACTION_CREATE_DOCUMENT).apply {
          addCategory(Intent.CATEGORY_OPENABLE)
          type = "application/pdf"
          putExtra(Intent.EXTRA_TITLE, "invoice.pdf")
  
          // Optionally, specify a URI for the directory that should be opened in
          // the system file picker before your app creates the document.
          putExtra(DocumentsContract.EXTRA_INITIAL_URI, pickerInitialUri)
      }
      startActivityForResult(intent, CREATE_FILE)
  }

  
  

• Getting the photo to process it: In order to get the photo as a Bitmap to process it, you can get an InputStream from the Uri and pass it to BitmapFactory.decodeResource or use the new ImageDecoder API on API 28+
  

  val bitmap = if (Build.VERSION.SDK_INT >= Build.VERSION_CODES.P) {
    ImageDecoder.decodeBitmap(ImageDecoder.createSource(contentResolver, uri))
  } else {
    contentResolver.openInputStream(uri)?.use { inputStream ->
      BitmapFactory.decodeStream(inputStream)
    }
  }

  
  

• Store a bitmap: You can save a bitmap as an image inside the pictures directory. In order to do, you have to use MediaStore API:
  

  private fun saveImageToStorage(
      bitmap: Bitmap,
      filename: String = "screenshot.jpg",
      mimeType: String =  "image/jpeg",
      directory: String = Environment.DIRECTORY_PICTURES,
      mediaContentUri: Uri = MediaStore.Images.Media.EXTERNAL_CONTENT_URI
  ) {
      val imageOutStream: OutputStream
      if (Build.VERSION.SDK_INT >= Build.VERSION_CODES.Q) {
          val values = ContentValues().apply {
              put(MediaStore.Images.Media.DISPLAY_NAME, filename)
              put(MediaStore.Images.Media.MIME_TYPE, mimeType)
              put(MediaStore.Images.Media.RELATIVE_PATH, directory)
          }
  
          contentResolver.run {
              val uri =
                  contentResolver.insert(mediaContentUri, values)
                      ?: return
              imageOutStream = openOutputStream(uri) ?: return
          }
      } else {
          val imagePath = Environment.getExternalStoragePublicDirectory(directory).absolutePath
          val image = File(imagePath, filename)
          imageOutStream = FileOutputStream(image)
      }
  
      imageOutStream.use { bitmap.compress(Bitmap.CompressFormat.JPEG, 100, it) }
  }

  
  
  
• How can create copy of a file: To create a copy of a file, we need to use openFileDescriptor. So, we'll pick a file from gallery and add it to your apps cache directory.
  
  

  val parcelFileDescriptor = context.contentResolver.openFileDescriptor(fileUri, "r", null)

  after that create instance of input stream:
  

  val inputStream = FileInputStream(parcelFileDescriptor.fileDescriptor)

  now provide a name to new file. You can give your own file name or you can get it from Uri as shown below:
  

  fun ContentResolver.getFileName(fileUri: Uri): String {
  
      var name = ""
      val returnCursor = this.query(fileUri, null, null, null, null)
      if (returnCursor != null) {
          val nameIndex = returnCursor.getColumnIndex(OpenableColumns.DISPLAY_NAME)
          returnCursor.moveToFirst()
          name = returnCursor.getString(nameIndex)
          returnCursor.close()
      }
  
      return name
  }

   create instance of file:
  

   val file = File(context.cacheDir, getFileName(context.contentResolver, fileUri))

  after that create instance of output stream to store file:
  

  val outputStream = FileOutputStream(file)

  The final step is to copy the contents of the original file to our newly created file in our cache directory. You can do this step in multiple ways, one easy way is to use the copy function from the IOUtils class(org.apache.commons.io.IOUtils).
  

  IOUtils.copy(inputStream, outputStream)

  now, you can use file to perform any action.
  

Conclusion

The Scoped Storage is good approach to make file secure. The important thing is that we can’t use the File APIs to directly access files anymore that's make file very secure. Now, we have to use Storage Access Framework for choosing files or folders and the MediaStore for media files.

I hope the approach shown in this post have been helpful to you and if you are facing any problem to implement it and you have any quires, please feel free to ask it from comment section below.