Commit Graph

116 Commits

Author SHA1 Message Date
Gary R Hook 6263b51eb3 crypto: ccp - Change ISR handler method for a v5 CCP
The CCP has the ability to perform several operations simultaneously,
but only one interrupt.  When implemented as a PCI device and using
MSI-X/MSI interrupts, use a tasklet model to service interrupts. By
disabling and enabling interrupts from the CCP, coupled with the
queuing that tasklets provide, we can ensure that all events
(occurring on the device) are recognized and serviced.

This change fixes a problem wherein 2 or more busy queues can cause
notification bits to change state while a (CCP) interrupt is being
serviced, but after the queue state has been evaluated. This results
in the event being 'lost' and the queue hanging, waiting to be
serviced. Since the status bits are never fully de-asserted, the
CCP never generates another interrupt (all bits zero -> one or more
bits one), and no further CCP operations will be executed.

Cc: <stable@vger.kernel.org> # 4.9.x+

Signed-off-by: Gary R Hook <gary.hook@amd.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2017-04-24 18:11:07 +08:00
Gary R Hook 7b537b24e7 crypto: ccp - Change ISR handler method for a v3 CCP
The CCP has the ability to perform several operations simultaneously,
but only one interrupt.  When implemented as a PCI device and using
MSI-X/MSI interrupts, use a tasklet model to service interrupts. By
disabling and enabling interrupts from the CCP, coupled with the
queuing that tasklets provide, we can ensure that all events
(occurring on the device) are recognized and serviced.

This change fixes a problem wherein 2 or more busy queues can cause
notification bits to change state while a (CCP) interrupt is being
serviced, but after the queue state has been evaluated. This results
in the event being 'lost' and the queue hanging, waiting to be
serviced. Since the status bits are never fully de-asserted, the
CCP never generates another interrupt (all bits zero -> one or more
bits one), and no further CCP operations will be executed.

Cc: <stable@vger.kernel.org> # 4.9.x+

Signed-off-by: Gary R Hook <gary.hook@amd.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2017-04-24 18:11:06 +08:00
Gary R Hook 116591fe3e crypto: ccp - Disable interrupts early on unload
Ensure that we disable interrupts first when shutting down
the driver.

Cc: <stable@vger.kernel.org> # 4.9.x+

Signed-off-by: Gary R Hook <ghook@amd.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2017-04-21 20:30:57 +08:00
Gary R Hook 56467cb11c crypto: ccp - Use only the relevant interrupt bits
Each CCP queue can product interrupts for 4 conditions:
operation complete, queue empty, error, and queue stopped.
This driver only works with completion and error events.

Cc: <stable@vger.kernel.org> # 4.9.x+

Signed-off-by: Gary R Hook <gary.hook@amd.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2017-04-21 20:30:56 +08:00
Gary R Hook 2d15839106 crypto: ccp - Rearrange structure members to minimize size
The AES GCM function (in ccp-ops) requires a fair amount of
stack space, which elicits a complaint when KASAN is enabled.
Rearranging and packing a few structures eliminates the
warning.

Signed-off-by: Gary R Hook <gary.hook@amd.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2017-04-05 21:58:35 +08:00
Gary R Hook 51de7dd02d crypto: ccp - Remove redundant cpu-to-le32 macros
Endianness is dealt with when the command descriptor is
copied into the command queue. Remove any occurrences of
cpu_to_le32() found elsewhere.

Signed-off-by: Gary R Hook <gary.hook@amd.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2017-04-05 21:58:34 +08:00
Herbert Xu c6dc060906 Merge git://git.kernel.org/pub/scm/linux/kernel/git/herbert/crypto-2.6
Merge the crypto tree to resolve conflict between caam changes.
2017-04-05 21:57:07 +08:00
Gary R Hook 36cf515b9b crypto: ccp - Enable support for AES GCM on v5 CCPs
A version 5 device provides the primitive commands
required for AES GCM. This patch adds support for
en/decryption.

Signed-off-by: Gary R Hook <gary.hook@amd.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2017-03-24 22:02:55 +08:00
Gary R Hook 990672d485 crypto: ccp - Enable 3DES function on v5 CCPs
Wire up support for Triple DES in ECB mode.

Signed-off-by: Gary R Hook <gary.hook@amd.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2017-03-24 22:02:55 +08:00
Gary R Hook ccebcf3f22 crypto: ccp - Add SHA-2 384- and 512-bit support
Incorporate 384-bit and 512-bit hashing for a version 5 CCP
device

Signed-off-by: Gary R Hook <gary.hook@amd.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2017-03-24 22:02:54 +08:00
Gary R Hook efc989fce8 crypto: ccp - Make some CCP DMA channels private
The CCP registers its queues as channels capable of handling
general DMA operations. The NTB driver will use DMA if
directed, but as public channels can be reserved for use in
asynchronous operations some channels should be held back
as private. Since the public/private determination is
handled at a device level, reserve the "other" (secondary)
CCP channels as private.

Add a module parameter that allows for override, to be
applied to all channels on all devices.

CC: <stable@vger.kernel.org> # 4.10.x-
Signed-off-by: Gary R Hook <gary.hook@amd.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2017-03-24 21:51:34 +08:00
Gary R Hook 7c468447f4 crypto: ccp - Assign DMA commands to the channel's CCP
The CCP driver generally uses a round-robin approach when
assigning operations to available CCPs. For the DMA engine,
however, the DMA mappings of the SGs are associated with a
specific CCP. When an IOMMU is enabled, the IOMMU is
programmed based on this specific device.

If the DMA operations are not performed by that specific
CCP then addressing errors and I/O page faults will occur.

Update the CCP driver to allow a specific CCP device to be
requested for an operation and use this in the DMA engine
support.

Cc: <stable@vger.kernel.org> # 4.9.x-
Signed-off-by: Gary R Hook <gary.hook@amd.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2017-03-16 17:43:08 +08:00
Gary R Hook 83d650ab78 crypto: ccp - Simplify some buffer management routines
The reverse-get/set functions can be simplified by
eliminating unused code.

Signed-off-by: Gary R Hook <gary.hook@amd.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2017-02-15 13:23:35 +08:00
Gary R Hook 4cdf101ef4 crypto: ccp - Update the command queue on errors
Move the command queue tail pointer when an error is
detected. Always return the error.

Signed-off-by: Gary R Hook <gary.hook@amd.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2017-02-15 13:23:34 +08:00
Gary R Hook a60496a0ca crypto: ccp - Change mode for detailed CCP init messages
The CCP initialization messages only need to be sent to
syslog in debug mode.

Signed-off-by: Gary R Hook <gary.hook@amd.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2017-02-15 13:23:34 +08:00
Gary R Hook f7cc02b3c3 crypto: ccp - Set the AES size field for all modes
Ensure that the size field is correctly populated for
all AES modes.

Signed-off-by: Gary R Hook <gary.hook@amd.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2017-02-15 13:20:19 +08:00
Gary R Hook e5da5c5667 crypto: ccp - Fix double add when creating new DMA command
Eliminate a double-add by creating a new list to manage
command descriptors when created; move the descriptor to
the pending list when the command is submitted.

Cc: <stable@vger.kernel.org>
Signed-off-by: Gary R Hook <gary.hook@amd.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2017-02-03 17:45:46 +08:00
Gary R Hook 500c0106e6 crypto: ccp - Fix DMA operations when IOMMU is enabled
An I/O page fault occurs when the IOMMU is enabled on a
system that supports the v5 CCP.  DMA operations use a
Request ID value that does not match what is expected by
the IOMMU, resulting in the I/O page fault.  Setting the
Request ID value to 0 corrects this issue.

Cc: <stable@vger.kernel.org>
Signed-off-by: Gary R Hook <gary.hook@amd.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2017-02-03 17:45:45 +08:00
Gary R Hook e6414b13ea crypto: ccp - Fix handling of RSA exponent on a v5 device
The exponent size in the ccp_op structure is in bits. A v5
CCP requires the exponent size to be in bytes, so convert
the size from bits to bytes when populating the descriptor.

The current code references the exponent in memory, but
these fields have not been set since the exponent is
actually store in the LSB. Populate the descriptor with
the LSB location (address).

Signed-off-by: Gary R Hook <gary.hook@amd.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2016-11-13 17:44:59 +08:00
Gary R Hook 103600ab96 crypto: ccp - Clean up the LSB slot allocation code
Fix a few problems revealed by testing: verify consistent
units, especially in public slot allocation. Percolate
some common initialization code up to a common routine.
Add some comments.

Signed-off-by: Gary R Hook <gary.hook@amd.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2016-10-25 11:08:23 +08:00
Gary R Hook ec9b70df75 crypto: ccp - remove unneeded code
Clean up patch for an unneeded structure member.

Signed-off-by: Gary R Hook <gary.hook@amd.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2016-10-25 11:08:21 +08:00
Gary R Hook fdd2cf9db1 crypto: ccp - change bitfield type to unsigned ints
Bit fields are not sensitive to endianness, so use
a transparent standard data type

Signed-off-by: Gary R Hook <gary.hook@amd.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2016-10-25 11:08:20 +08:00
Wei Yongjun ff4f44de44 crypto: ccp - Fix non static symbol warning
Fixes the following sparse warning:

drivers/crypto/ccp/ccp-dev.c:44:6: warning:
 symbol 'ccp_error_codes' was not declared. Should it be static?

Signed-off-by: Wei Yongjun <weiyongjun1@huawei.com>
Acked-by: Gary R Hook <gary.hook@amd.com>
Acked-by: Gary R Hook <gary.hook@amd.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2016-10-25 11:08:17 +08:00
Gary R Hook 3cf799680d crypto: ccp - change type of struct member lsb to signed
The lsb field uses a value of -1 to indicate that it
is unassigned. Therefore type must be a signed int.

Signed-off-by: Gary R Hook <gary.hook@amd.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2016-10-21 11:03:46 +08:00
Gary R Hook 81422badb3 crypto: ccp - Make syslog errors human-readable
Add human-readable strings to log messages about CCP errors

Signed-off-by: Gary R Hook <gary.hook@amd.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2016-10-02 22:33:45 +08:00
Gary R Hook 9ddb9dc6be crypto: ccp - clean up data structure
Change names of data structure instances.  Add const
keyword where appropriate.  Add error handling path.

Signed-off-by: Gary R Hook <gary.hook@amd.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2016-10-02 22:33:44 +08:00
Wei Yongjun 7514e36888 crypto: ccp - Fix return value check in ccp_dmaengine_register()
Fix the retrn value check which testing the wrong variable
in ccp_dmaengine_register().

Fixes: 58ea8abf49 ("crypto: ccp - Register the CCP as a DMA resource")
Signed-off-by: Wei Yongjun <weiyongjun1@huawei.com>
Acked-by: Gary R Hook <gary.hook@amd.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2016-09-22 18:32:47 +08:00
Wei Yongjun 664f570a9c crypto: ccp - use kmem_cache_zalloc instead of kmem_cache_alloc/memset
Using kmem_cache_zalloc() instead of kmem_cache_alloc() and memset().

Signed-off-by: Wei Yongjun <weiyongjun1@huawei.com>
Acked-by: Gary R Hook <gary.hook@amd.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2016-09-22 18:27:37 +08:00
Quentin Lambert ba22a1e2aa crypto: ccp - add missing release in ccp_dmaengine_register
ccp_dmaengine_register used to return with an error code before
releasing all resource. This patch adds a jump to the appropriate label
ensuring that the resources are properly released before returning.

This issue was found with Hector.

Signed-off-by: Quentin Lambert <lambert.quentin@gmail.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2016-09-07 21:08:31 +08:00
Wei Yongjun dabc7904a7 crypto: ccp - Fix non static symbol warning
Fixes the following sparse warning:

drivers/crypto/ccp/ccp-dev.c:62:14: warning:
 symbol 'ccp_increment_unit_ordinal' was not declared. Should it be static?

Signed-off-by: Wei Yongjun <weiyj.lk@gmail.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2016-08-16 17:20:18 +08:00
Gary R Hook e14e7d1267 crypto: ccp - Enable use of the additional CCP
A second CCP is available, identical to the first, with
its ownn PCI ID. Make it available for use by the crypto
subsystem, as well as for DMA activity and random
number generation.

This device is not pre-configured at at boot time. The
driver must configure it (during the probe) for use.

Signed-off-by: Gary R Hook <gary.hook@amd.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2016-08-09 18:47:18 +08:00
Gary R Hook 99d90b2ebd crypto: ccp - Enable DMA service on a v5 CCP
Every CCP is capable of providing general DMA services.
Register the device as a provider.

Signed-off-by: Gary R Hook <gary.hook@amd.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2016-08-09 18:47:18 +08:00
Gary R Hook 084935b208 crypto: ccp - Add support for the RNG in a version 5 CCP
Signed-off-by: Gary R Hook <gary.hook@amd.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2016-08-09 18:47:17 +08:00
Gary R Hook 4b394a232d crypto: ccp - Let a v5 CCP provide the same function as v3
Enable equivalent function on a v5 CCP. Add support for a
version 5 CCP which enables AES/XTS/SHA services. Also,
more work on the data structures to virtualize
functionality.

Signed-off-by: Gary R Hook <gary.hook@amd.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2016-08-09 18:47:16 +08:00
Gary R Hook bb4e89b34d crypto: ccp - Refactor code to enable checks for queue space.
Available queue space is used to decide (by counting free slots)
if we have to put a command on hold or if it can be sent
to the engine immediately.

Signed-off-by: Gary R Hook <gary.hook@amd.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2016-08-09 18:47:10 +08:00
Gary R Hook 8256e68311 crypto: ccp - Refactor code supporting the CCP's RNG
Make the RNG support code common (where possible) in
preparation for adding a v5 device.

Signed-off-by: Gary R Hook <gary.hook@amd.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2016-08-09 18:47:09 +08:00
Gary R Hook 58a690b701 crypto: ccp - Refactor the storage block allocation code
Move the KSB access/management functions to the v3
device file, and add function pointers to the actions
structure. At the operations layer all of the references
to the storage block will be generic (virtual). This is
in preparation for a version 5 device, in which the
private storage block is managed differently.

Signed-off-by: Gary R Hook <gary.hook@amd.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2016-08-09 18:47:08 +08:00
Gary R Hook 956ee21a6d crypto: ccp - Refactoring: symbol cleanup
Form and use of the local storage block in the CCP is
particular to the device version. Much of the code that
accesses the storage block can treat it as a virtual
resource, and will under go some renaming. Device-specific
access to the memory will be moved into device file.
Service functions will be added to the actions
structure.

Signed-off-by: Gary R Hook <gary.hook@amd.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2016-08-09 18:47:08 +08:00
Gary R Hook a43eb98507 crypto: ccp - Shorten the fields of the action structure
Use more concise field names; "perform_" is too verbose.

Signed-off-by: Gary R Hook <gary.hook@amd.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2016-08-09 18:47:07 +08:00
Gary R Hook fba8855cb2 crypto: ccp - Abstract PCI info for the CCP
Device-specific values for the BAR and offset should be found
in the version data structure.

Signed-off-by: Gary R Hook <gary.hook@amd.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2016-08-09 18:47:05 +08:00
Gary R Hook fa242e80c7 crypto: ccp - Fix non-conforming comment style
Adhere to the cryptodev comment convention.

Signed-off-by: Gary R Hook <gary.hook@amd.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2016-08-09 18:47:04 +08:00
Herbert Xu 241118de58 crypto: ccp - Use skcipher for fallback
This patch replaces use of the obsolete ablkcipher with skcipher.

Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2016-07-01 23:45:08 +08:00
Tom Lendacky ab6a11a7c8 crypto: ccp - Fix AES XTS error for request sizes above 4096
The ccp-crypto module for AES XTS support has a bug that can allow requests
greater than 4096 bytes in size to be passed to the CCP hardware. The CCP
hardware does not support request sizes larger than 4096, resulting in
incorrect output. The request should actually be handled by the fallback
mechanism instantiated by the ccp-crypto module.

Add a check to insure the request size is less than or equal to the maximum
supported size and use the fallback mechanism if it is not.

Cc: <stable@vger.kernel.org> # 3.14.x-
Signed-off-by: Tom Lendacky <thomas.lendacky@amd.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2016-05-25 18:33:47 +08:00
Julia Lawall bc197b2a9c crypto: ccp - constify ccp_actions structure
The ccp_actions structure is never modified, so declare it as const.

Done with the help of Coccinelle.

Signed-off-by: Julia Lawall <Julia.Lawall@lip6.fr>
Acked-by: Gary Hook <gary.hook@amd.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2016-05-03 16:10:14 +08:00
Herbert Xu 6f6438975d Merge git://git.kernel.org/pub/scm/linux/kernel/git/herbert/crypto-2.6
Merge the crypto tree to pull in the qat adf_init_pf_wq change.
2016-04-28 09:13:03 +08:00
Gary R Hook b3c2fee5d6 crypto: ccp - Ensure all dependencies are specified
A DMA_ENGINE requires DMADEVICES in Kconfig

Signed-off-by: Gary R Hook <gary.hook@amd.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2016-04-25 19:14:51 +08:00
Gary R Hook 58ea8abf49 crypto: ccp - Register the CCP as a DMA resource
The CCP has the ability to provide DMA services to the
kernel using pass-through mode of the device. Register
these services as general purpose DMA channels.

Changes since v2:
- Add a Signed-off-by

Changes since v1:
- Allocate memory for a string in ccp_dmaengine_register
- Ensure register/unregister calls are properly ordered
- Verified all changed files are listed in the diffstat
- Undo some superfluous changes
- Added a cc:

Signed-off-by: Gary R Hook <gary.hook@amd.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2016-04-20 17:50:06 +08:00
Mike Galbraith 7587c40754 crypto: ccp - Fix RT breaking #include <linux/rwlock_types.h>
Direct include of rwlock_types.h breaks RT, use spinlock_types.h instead.

Fixes: 553d2374db crypto: ccp - Support for multiple CCPs
Signed-off-by: Mike Galbraith <umgwanakikbuti@gmail.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2016-04-15 22:35:44 +08:00
Tom Lendacky f709b45ec4 crypto: ccp - Prevent information leakage on export
Prevent information from leaking to userspace by doing a memset to 0 of
the export state structure before setting the structure values and copying
it. This prevents un-initialized padding areas from being copied into the
export area.

Cc: <stable@vger.kernel.org> # 3.14.x-
Reported-by: Ben Hutchings <ben@decadent.org.uk>
Signed-off-by: Tom Lendacky <thomas.lendacky@amd.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2016-04-15 22:13:56 +08:00
Gary R Hook 03a6f29000 crypto: ccp - fix lock acquisition code
This patch simplifies an unneeded read-write lock.

Signed-off-by: Gary R Hook <gary.hook@amd.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2016-03-17 19:09:02 +08:00